[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:39 dist-test-slave-pdnb krb5kdc[7723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:39 dist-test-slave-pdnb krb5kdc[7723](info): set up 2 sockets
Jun 19 02:40:39 dist-test-slave-pdnb krb5kdc[7723](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 02:40:40.637619 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:40 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:40.756326 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37233
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:35997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:40.837697 7736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:40.837934 7736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:40.837970 7736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:40.839118 7736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:40.839187 7736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:40.839212 7736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:40.839241 7736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:40.839265 7736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:40.840696 7736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:35997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:40.840999 7736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:40.841215 7736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:40.846599 7744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:40.846608 7742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:40.846616 7741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:40.846637 7736 server_base.cc:1034] running on GCE node
I20260619 02:40:40.847087 7736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:40.847380 7736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:40.848579 7736 hybrid_clock.cc:648] HybridClock initialized: now 1781836840848572 us; error 38 us; skew 500 ppm
Jun 19 02:40:40 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836840, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:40.851142 7736 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:40.851591 7736 webserver.cc:460] Webserver started at http://127.7.137.62:44677/ using document root <none> and password file <none>
I20260619 02:40:40.851771 7736 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:40.851817 7736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:40.851907 7736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:40.852982 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "ec60c9da2bfd454ea04a78afce2b806f"
format_stamp: "Formatted at 2026-06-19 02:40:40 on dist-test-slave-pdnb"
server_key: "29db9335f70d37e2897e7b9470f3af23"
server_key_iv: "bc7e624e49c38c35647e0371c5de7c11"
server_key_version: "encryptionkey@0"
I20260619 02:40:40.853287 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "ec60c9da2bfd454ea04a78afce2b806f"
format_stamp: "Formatted at 2026-06-19 02:40:40 on dist-test-slave-pdnb"
server_key: "29db9335f70d37e2897e7b9470f3af23"
server_key_iv: "bc7e624e49c38c35647e0371c5de7c11"
server_key_version: "encryptionkey@0"
I20260619 02:40:40.854609 7736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:40.855301 7751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:40.855490 7736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:40.855569 7736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "ec60c9da2bfd454ea04a78afce2b806f"
format_stamp: "Formatted at 2026-06-19 02:40:40 on dist-test-slave-pdnb"
server_key: "29db9335f70d37e2897e7b9470f3af23"
server_key_iv: "bc7e624e49c38c35647e0371c5de7c11"
server_key_version: "encryptionkey@0"
I20260619 02:40:40.855623 7736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:40.873134 7736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:40.875211 7736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:40.875314 7736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:40.878961 7736 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37233
I20260619 02:40:40.879036 7803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37233 every 8 connection(s)
I20260619 02:40:40.879361 7736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:40.880267 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:40.881047 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 7736
I20260619 02:40:40.881125 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:40.881739 7716 external_mini_cluster.cc:1428] Setting key 03f1b91fdd271dc8a35451be5ad98509
I20260619 02:40:40.883074 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f: Bootstrap starting.
I20260619 02:40:40.883634 7804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:40.883922 7804 log.cc:824] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:40.884610 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f: No bootstrap required, opened a new log
I20260619 02:40:40.886350 7804 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } }
I20260619 02:40:40.886503 7804 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:40.886550 7804 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60c9da2bfd454ea04a78afce2b806f, State: Initialized, Role: FOLLOWER
I20260619 02:40:40.886720 7804 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } }
I20260619 02:40:40.886818 7804 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:40.886865 7804 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:40.886906 7804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:40.887444 7804 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } }
I20260619 02:40:40.887527 7804 leader_election.cc:302] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec60c9da2bfd454ea04a78afce2b806f; no voters:
I20260619 02:40:40.887708 7804 leader_election.cc:288] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:40.887732 7809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:40.887853 7809 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [term 1 LEADER]: Becoming Leader. State: Replica: ec60c9da2bfd454ea04a78afce2b806f, State: Running, Role: LEADER
I20260619 02:40:40.887959 7804 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:40.887939 7809 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } }
I20260619 02:40:40.888278 7809 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec60c9da2bfd454ea04a78afce2b806f. Latest consensus state: current_term: 1 leader_uuid: "ec60c9da2bfd454ea04a78afce2b806f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } } }
I20260619 02:40:40.888357 7809 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:40.888285 7811 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec60c9da2bfd454ea04a78afce2b806f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec60c9da2bfd454ea04a78afce2b806f" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37233 } } }
I20260619 02:40:40.888521 7811 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:40.888567 7814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:40.889117 7814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:40.890825 7814 catalog_manager.cc:1347] Generated new cluster ID: 8d199730c3cd415daf7ff0a03b0e8e89
I20260619 02:40:40.890882 7814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:40 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:40.901046 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:40.882620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.7.137.62:37233)
0619 02:40:40.882771 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:40.882782 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:40.883032 (+ 250us) server_negotiation.cc:407] Connection header received
0619 02:40:40.885163 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:40.885169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:40.885212 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:40.885348 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:40.886832 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:40.888335 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:40.889910 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:40.890422 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:40.896086 (+ 5664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:40.896187 (+ 101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:40.896190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:40.896190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:40.896199 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:40.898775 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:40.899248 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:40.899251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:40.899251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:40.899344 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:40.899596 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:40.899597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:40.899597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:40.899868 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:40.900603 (+ 735us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:40.900666 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:40:40.900823 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260619 02:40:40.903317 7814 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:40.904107 7814 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:40.908850 7814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec60c9da2bfd454ea04a78afce2b806f: Generated new TSK 0
I20260619 02:40:40.909047 7814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:40.934965 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:35997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:41.016069 7830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:41.016296 7830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:41.016336 7830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:41.017560 7830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:41.017627 7830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:41.017673 7830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:41.019165 7830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:35997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:41.019431 7830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:41.019655 7830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:41.024587 7838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:41.024668 7835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:41.024607 7836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:41.024930 7830 server_base.cc:1034] running on GCE node
I20260619 02:40:41.025148 7830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:41.025404 7830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:41.026559 7830 hybrid_clock.cc:648] HybridClock initialized: now 1781836841026540 us; error 36 us; skew 500 ppm
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:41.029000 7830 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:41.029448 7830 webserver.cc:460] Webserver started at http://127.7.137.1:45833/ using document root <none> and password file <none>
I20260619 02:40:41.029667 7830 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:41.029738 7830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:41.029887 7830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:41.030923 7830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "a28c881aabff4828a12d63b8f1093087"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "f5360c41aea20ec6f9ac726b2d77bb0d"
server_key_iv: "c81baade81907751b2e58aac83f6a2b8"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.031297 7830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "a28c881aabff4828a12d63b8f1093087"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "f5360c41aea20ec6f9ac726b2d77bb0d"
server_key_iv: "c81baade81907751b2e58aac83f6a2b8"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.032562 7830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:41.033398 7845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.033599 7830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:41.033697 7830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "a28c881aabff4828a12d63b8f1093087"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "f5360c41aea20ec6f9ac726b2d77bb0d"
server_key_iv: "c81baade81907751b2e58aac83f6a2b8"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.033782 7830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:41.049628 7830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:41.051484 7830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:41.051659 7830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:41.051896 7830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:41.052205 7830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:41.052260 7830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.052294 7830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:41.052321 7830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.058135 7830 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:46415
I20260619 02:40:41.058173 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:46415 every 8 connection(s)
I20260619 02:40:41.058511 7830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:41.059921 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 7830
I20260619 02:40:41.060032 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:41.060199 7716 external_mini_cluster.cc:1428] Setting key df1c266b848824ecd3865841075d9127
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
I20260619 02:40:41.069159 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.059258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:44727 (local address 127.7.137.62:37233)
0619 02:40:41.059304 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:41.059306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:41.059535 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:40:41.060033 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:41.060035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:41.060046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:41.060103 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:41.061241 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.061757 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.063247 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.063603 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.063674 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.065895 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:41.065896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:41.065897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:41.065906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:41.067617 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.068027 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.068028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.068028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.068092 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.068337 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.068338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.068338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.068487 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:41.068558 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:41.068944 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 02:40:41.069054 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:41.069207 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.059251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37233 (local address 127.7.137.1:44727)
0619 02:40:41.059409 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:41.059445 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:41.059912 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:41.060175 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:41.060179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:41.060952 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:41.061145 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.061150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.061853 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.061854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.063159 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.063161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.063693 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.063694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.063806 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.064375 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:41.064387 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:41.065802 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:41.067706 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.067711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.067713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.067949 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.068168 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.068168 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.068168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.068266 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.068556 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:41.068560 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:41.068714 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 02:40:41.068918 (+ 204us) client_negotiation.cc:240] Negotiation successful
0619 02:40:41.069046 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260619 02:40:41.069787 7959 heartbeater.cc:347] Connected to a master server at 127.7.137.62:37233
I20260619 02:40:41.069886 7959 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:41.070061 7959 heartbeater.cc:511] Master 127.7.137.62:37233 requested a full tablet report, sending...
I20260619 02:40:41.070681 7768 ts_manager.cc:194] Registered new tserver with Master: a28c881aabff4828a12d63b8f1093087 (127.7.137.1:46415)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
I20260619 02:40:41.071431 7768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:44727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:41.085520 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:35997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:41.164326 7964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:41.164554 7964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:41.164590 7964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:41.165727 7964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:41.165794 7964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:41.165838 7964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:41.167232 7964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:35997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:41.167500 7964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:41.167721 7964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:41.172967 7972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:41.172976 7970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:41.173004 7964 server_base.cc:1034] running on GCE node
W20260619 02:40:41.173002 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:41.173372 7964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:41.173570 7964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:41.174707 7964 hybrid_clock.cc:648] HybridClock initialized: now 1781836841174683 us; error 39 us; skew 500 ppm
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:41.177291 7964 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:41.177673 7964 webserver.cc:460] Webserver started at http://127.7.137.2:34565/ using document root <none> and password file <none>
I20260619 02:40:41.177846 7964 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:41.177886 7964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:41.177973 7964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:41.178937 7964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "bf5d1f88301c45fd8ff84eb55bd663b1"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84f9a7f6a932429a42dc91fc1fd6fb12"
server_key_iv: "8156685c3e5bf6e0b01396add2c5447f"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.179226 7964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "bf5d1f88301c45fd8ff84eb55bd663b1"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84f9a7f6a932429a42dc91fc1fd6fb12"
server_key_iv: "8156685c3e5bf6e0b01396add2c5447f"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.180472 7964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:41.181176 7979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.181347 7964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.181456 7964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "bf5d1f88301c45fd8ff84eb55bd663b1"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84f9a7f6a932429a42dc91fc1fd6fb12"
server_key_iv: "8156685c3e5bf6e0b01396add2c5447f"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.181529 7964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:41.195053 7964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:41.197289 7964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:41.197451 7964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:41.197670 7964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:41.197953 7964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:41.198005 7964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.198045 7964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:41.198072 7964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.204003 7964 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:40999
I20260619 02:40:41.204051 8092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:40999 every 8 connection(s)
I20260619 02:40:41.204344 7964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:41.209980 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 7964
I20260619 02:40:41.210086 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:41.210253 7716 external_mini_cluster.cc:1428] Setting key aed38ddc831868b068f6bbd635fcd138
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:41.213956 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.205081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:54435 (local address 127.7.137.62:37233)
0619 02:40:41.205165 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:41.205167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:41.205349 (+ 182us) server_negotiation.cc:407] Connection header received
0619 02:40:41.205878 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:41.205880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:41.205892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:41.205954 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:41.206888 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.207410 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.208395 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.208600 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.208678 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.210942 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:41.210943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:41.210943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:41.210950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:41.212359 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.212874 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.212875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.212875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.212927 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.213234 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.213234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.213235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.213380 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:41.213451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:41.213707 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:40:41.213803 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:40:41.213992 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.205039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37233 (local address 127.7.137.2:54435)
0619 02:40:41.205225 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:41.205272 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:41.205771 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:41.206032 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:41.206036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:41.206602 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:41.206786 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.206790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.207508 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.207509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.208305 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.208306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.208682 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.208683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.208822 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.209283 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:41.209291 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:41.210834 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:41.212442 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.212449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.212452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.212798 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.212995 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.212996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.212996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.213152 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.213450 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:41.213455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:41.213576 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:40:41.213663 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:40:41.213785 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260619 02:40:41.214633 8093 heartbeater.cc:347] Connected to a master server at 127.7.137.62:37233
I20260619 02:40:41.214752 8093 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:41.214931 8093 heartbeater.cc:511] Master 127.7.137.62:37233 requested a full tablet report, sending...
I20260619 02:40:41.215323 7768 ts_manager.cc:194] Registered new tserver with Master: bf5d1f88301c45fd8ff84eb55bd663b1 (127.7.137.2:40999)
I20260619 02:40:41.215739 7768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:54435
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:41.233327 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:35997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:41.314616 8098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:41.314812 8098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:41.314836 8098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:41.315971 8098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:41.316012 8098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:41.316063 8098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:41.317615 8098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:35997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:37233
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:41.317853 8098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:41.318084 8098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:41.323213 8106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:41.323243 8104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:41.323436 8103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:41.323753 8098 server_base.cc:1034] running on GCE node
I20260619 02:40:41.323961 8098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:41.324189 8098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:41.325347 8098 hybrid_clock.cc:648] HybridClock initialized: now 1781836841325307 us; error 50 us; skew 500 ppm
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:41.327849 8098 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:41.328280 8098 webserver.cc:460] Webserver started at http://127.7.137.3:37317/ using document root <none> and password file <none>
I20260619 02:40:41.328521 8098 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:41.328589 8098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:41.328786 8098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:41.329844 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "107966b2165f40b792cba80732316518"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84b60f2174ec254e4f138b19eec9a5db"
server_key_iv: "4caf94691b5c888b4b41ba0161ffb88d"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.330214 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "107966b2165f40b792cba80732316518"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84b60f2174ec254e4f138b19eec9a5db"
server_key_iv: "4caf94691b5c888b4b41ba0161ffb88d"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.331503 8098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:41.332194 8113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.332348 8098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:41.332445 8098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "107966b2165f40b792cba80732316518"
format_stamp: "Formatted at 2026-06-19 02:40:41 on dist-test-slave-pdnb"
server_key: "84b60f2174ec254e4f138b19eec9a5db"
server_key_iv: "4caf94691b5c888b4b41ba0161ffb88d"
server_key_version: "encryptionkey@0"
I20260619 02:40:41.332525 8098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:41.357100 8098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:41.359515 8098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:41.359673 8098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:41.359895 8098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:41.360224 8098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:41.360280 8098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.360320 8098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:41.360348 8098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:41.366364 8098 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:44597
I20260619 02:40:41.366477 8226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:44597 every 8 connection(s)
I20260619 02:40:41.366751 8098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:41.368007 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8098
I20260619 02:40:41.368079 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:41.368268 7716 external_mini_cluster.cc:1428] Setting key ae9c250b5ec60f646539a133c4e38ff1
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:41.377132 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.367483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:41581 (local address 127.7.137.62:37233)
0619 02:40:41.367555 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:41.367557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:41.367727 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:40:41.368290 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:41.368291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:41.368304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:41.368366 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:41.369559 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.370066 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.370991 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.371173 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.371225 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.373297 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:41.373298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:41.373299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:41.373305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:41.374459 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.376036 (+ 1577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.376038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.376038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.376149 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.376422 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.376423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.376424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.376549 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:41.376620 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:41.376955 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 02:40:41.377039 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:40:41.377323 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.367460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37233 (local address 127.7.137.3:41581)
0619 02:40:41.367599 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:41.367641 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:41.368180 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:41.368483 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:41.368486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:41.369296 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:41.369489 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.369494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.370139 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.370140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.370907 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:41.370909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.371239 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.371240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:41.371335 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.371837 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:41.371846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:41.373205 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:41.374523 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.374529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.374530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.375202 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.376229 (+ 1027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:41.376230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:41.376230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:41.376340 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:41.376624 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:41.376629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:41.376815 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 02:40:41.376926 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:40:41.377087 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260619 02:40:41.377864 8227 heartbeater.cc:347] Connected to a master server at 127.7.137.62:37233
I20260619 02:40:41.377964 8227 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:41.378166 8227 heartbeater.cc:511] Master 127.7.137.62:37233 requested a full tablet report, sending...
I20260619 02:40:41.378556 7768 ts_manager.cc:194] Registered new tserver with Master: 107966b2165f40b792cba80732316518 (127.7.137.3:44597)
I20260619 02:40:41.378906 7768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:41581
I20260619 02:40:41.380278 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:41.403904 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.396237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.7.137.1:46415)
0619 02:40:41.396397 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:41.396407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:41.396419 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:40:41.396444 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:41.396446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:41.396460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:41.396533 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:41.397232 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.397992 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.398736 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.399066 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.399121 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.401370 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:41.401377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:41.401377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:41.401383 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:41.402776 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.403089 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.403092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.403092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.403142 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.403358 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.403359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.403359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.403513 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:41.403615 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:41.403739 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:40:41.403812 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260619 02:40:41.404278 7873 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43506
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[7723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:41.426120 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:41.419221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.7.137.1:46415)
0619 02:40:41.419296 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:41.419298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:41.419380 (+ 82us) server_negotiation.cc:407] Connection header received
0619 02:40:41.419485 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:41.419486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:41.419499 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:41.419572 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:41.420111 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.420607 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.421404 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:41.421615 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:41.421691 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:41.423980 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:41.423981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:41.423982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:41.423988 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:41.425234 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.425534 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.425535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.425535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.425582 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:41.425738 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:41.425738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:41.425739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:41.425828 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:41.425918 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:41.425992 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:40:41.426053 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:40:41.426676 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 7830
I20260619 02:40:41.431118 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 7964
I20260619 02:40:41.436630 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8098
I20260619 02:40:41.441027 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 7736
2026-06-19T02:40:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1846 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[8241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[8241](info): set up 2 sockets
Jun 19 02:40:41 dist-test-slave-pdnb krb5kdc[8241](info): commencing operation
krb5kdc: starting...
W20260619 02:40:42.484242 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:42 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:42.595611 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:34917
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:34917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:42.674675 8254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:42.674880 8254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:42.674901 8254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:42.676072 8254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:42.676117 8254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:42.676132 8254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:42.676146 8254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:42.676158 8254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:42.677656 8254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:34917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:34917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:42.677899 8254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:42.678138 8254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:42.683082 8259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:42.683080 8262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:42.683086 8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:42.683367 8254 server_base.cc:1034] running on GCE node
I20260619 02:40:42.683653 8254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:42.683902 8254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:42.685040 8254 hybrid_clock.cc:648] HybridClock initialized: now 1781836842685019 us; error 37 us; skew 500 ppm
Jun 19 02:40:42 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:42.687391 8254 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:42.687855 8254 webserver.cc:460] Webserver started at http://127.7.137.62:44225/ using document root <none> and password file <none>
I20260619 02:40:42.688098 8254 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:42.688166 8254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:42.688297 8254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:42.689404 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "68ade473fcce3fa60eff3630e66521fb"
server_key_iv: "30e8a02d84c712dd106a955bd124805c"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.689755 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "68ade473fcce3fa60eff3630e66521fb"
server_key_iv: "30e8a02d84c712dd106a955bd124805c"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.691062 8254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:42.691860 8269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:42.692070 8254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:42.692166 8254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "68ade473fcce3fa60eff3630e66521fb"
server_key_iv: "30e8a02d84c712dd106a955bd124805c"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.692246 8254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:42.716271 8254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:42.718364 8254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:42.718545 8254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:42.722688 8254 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:34917
I20260619 02:40:42.722728 8321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:34917 every 8 connection(s)
I20260619 02:40:42.723110 8254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:42.723940 8322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:42.726199 8322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718: Bootstrap starting.
I20260619 02:40:42.726800 8322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:42.727092 8322 log.cc:824] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:42.727747 8322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718: No bootstrap required, opened a new log
I20260619 02:40:42.729463 8322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } }
I20260619 02:40:42.729604 8322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:42.729627 8322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b6a10f14ba4a1db3ada6c1b3f3f718, State: Initialized, Role: FOLLOWER
I20260619 02:40:42.729799 8322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } }
I20260619 02:40:42.729867 8322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:42.729909 8322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:42.729957 8322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:42.730163 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8254
I20260619 02:40:42.730254 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:42.730414 7716 external_mini_cluster.cc:1428] Setting key 4287ce59d6e4158c24d51c1acc4f0bd1
I20260619 02:40:42.730597 8322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } }
I20260619 02:40:42.730708 8322 leader_election.cc:302] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b6a10f14ba4a1db3ada6c1b3f3f718; no voters:
I20260619 02:40:42.730892 8322 leader_election.cc:288] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:42.730952 8325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:42.731081 8325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [term 1 LEADER]: Becoming Leader. State: Replica: 95b6a10f14ba4a1db3ada6c1b3f3f718, State: Running, Role: LEADER
I20260619 02:40:42.731146 8322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:42.731166 8325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } }
I20260619 02:40:42.731526 8328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } } }
I20260619 02:40:42.731683 8328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:42.732025 8329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95b6a10f14ba4a1db3ada6c1b3f3f718. Latest consensus state: current_term: 1 leader_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b6a10f14ba4a1db3ada6c1b3f3f718" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34917 } } }
I20260619 02:40:42.732093 8329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:42.732247 8331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:42.732808 8331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:42.734377 8331 catalog_manager.cc:1347] Generated new cluster ID: 3d5d2046034b41f28411c6e768072ca6
I20260619 02:40:42.734440 8331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:42 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:42.742296 8331 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:42.742722 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:42.730854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.7.137.62:34917)
0619 02:40:42.731080 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:42.731090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:42.731631 (+ 541us) server_negotiation.cc:407] Connection header received
0619 02:40:42.731846 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:42.731854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:42.731927 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:42.732045 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:42.733066 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.734318 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.735370 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.735986 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.736085 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:42.738892 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:42.738896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:42.738896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:42.738908 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:42.740961 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:42.741365 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:42.741368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:42.741369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:42.741493 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:42.741704 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:42.741705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:42.741706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:42.742004 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:42.742161 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:42.742284 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 02:40:42.742501 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260619 02:40:42.743466 8331 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:42.756508 8331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95b6a10f14ba4a1db3ada6c1b3f3f718: Generated new TSK 0
I20260619 02:40:42.756699 8331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:42.782889 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:42.862207 8348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:42.862437 8348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:42.862474 8348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:42.863620 8348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:42.863687 8348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:42.863734 8348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:42.865286 8348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:42.865543 8348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:42.865775 8348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:42.870890 8354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:42.870895 8356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:42.871054 8348 server_base.cc:1034] running on GCE node
W20260619 02:40:42.870899 8353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:42.871315 8348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:42.871567 8348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:42.872762 8348 hybrid_clock.cc:648] HybridClock initialized: now 1781836842872747 us; error 25 us; skew 500 ppm
Jun 19 02:40:42 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:42.875166 8348 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:42.875597 8348 webserver.cc:460] Webserver started at http://127.7.137.1:44855/ using document root <none> and password file <none>
I20260619 02:40:42.875837 8348 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:42.875905 8348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:42.876070 8348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:42.877184 8348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "a02a80f1971415c2ecf87f71a6b01602"
server_key_iv: "acfd5add8a721db8787384da6297bf88"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.877545 8348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "a02a80f1971415c2ecf87f71a6b01602"
server_key_iv: "acfd5add8a721db8787384da6297bf88"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.878827 8348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:42.879742 8363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:42.879897 8348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:42.880018 8348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f"
format_stamp: "Formatted at 2026-06-19 02:40:42 on dist-test-slave-pdnb"
server_key: "a02a80f1971415c2ecf87f71a6b01602"
server_key_iv: "acfd5add8a721db8787384da6297bf88"
server_key_version: "encryptionkey@0"
I20260619 02:40:42.880106 8348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:42.893205 8348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:42.894946 8348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:42.895097 8348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:42.895308 8348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:42.895602 8348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:42.895656 8348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:42.895696 8348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:42.895723 8348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:42.901470 8348 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:39157
I20260619 02:40:42.901516 8476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:39157 every 8 connection(s)
I20260619 02:40:42.901816 8348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:42.907781 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8348
I20260619 02:40:42.907878 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:42.908049 7716 external_mini_cluster.cc:1428] Setting key 8a00aadbbd3e3fe8c6d2555b8c9a3c28
Jun 19 02:40:42 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:42.911537 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:42.902508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:43683 (local address 127.7.137.62:34917)
0619 02:40:42.902577 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:42.902579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:42.902777 (+ 198us) server_negotiation.cc:407] Connection header received
0619 02:40:42.903752 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:42.903753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:42.903768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:42.903826 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:42.904478 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.905063 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.906082 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.906290 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.906362 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:42.908745 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:42.908746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:42.908746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:42.908754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:42.910169 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:42.910564 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:42.910565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:42.910566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:42.910635 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:42.910854 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:42.910855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:42.910855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:42.911004 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:42.911079 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:42.911296 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:40:42.911398 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:40:42.911566 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:42.902490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34917 (local address 127.7.137.1:43683)
0619 02:40:42.902658 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:42.902692 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:42.903638 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:42.903900 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:42.903903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:42.904148 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:42.904336 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:42.904340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.905157 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.905158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:42.905975 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:42.905977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:42.906381 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:42.906382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:42.906508 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:42.907018 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:42.907027 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:42.908649 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:42.910255 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:42.910260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:42.910262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:42.910485 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:42.910701 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:42.910701 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:42.910702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:42.910795 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:42.911070 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:42.911074 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:42.911188 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:40:42.911269 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:40:42.911394 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260619 02:40:42.912214 8477 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34917
I20260619 02:40:42.912328 8477 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:42.912540 8477 heartbeater.cc:511] Master 127.7.137.62:34917 requested a full tablet report, sending...
I20260619 02:40:42.913158 8286 ts_manager.cc:194] Registered new tserver with Master: b1d5353d6e6d45d3bcdd6fa2fd41697f (127.7.137.1:39157)
I20260619 02:40:42.914173 8286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:43683
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:42.933034 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:43.013928 8482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:43.014132 8482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:43.014156 8482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:43.015329 8482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:43.015375 8482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:43.015403 8482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:43.017594 8482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:43.017836 8482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:43.018092 8482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:43.023465 8490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:43.023418 8488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:43.023630 8482 server_base.cc:1034] running on GCE node
W20260619 02:40:43.023418 8487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:43.023897 8482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:43.024140 8482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:43.025322 8482 hybrid_clock.cc:648] HybridClock initialized: now 1781836843025309 us; error 29 us; skew 500 ppm
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:43.027882 8482 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:43.028266 8482 webserver.cc:460] Webserver started at http://127.7.137.2:36609/ using document root <none> and password file <none>
I20260619 02:40:43.028484 8482 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:43.028527 8482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:43.028620 8482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:43.029701 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "685315caecd64ef890348ab3ad59753c"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "c93f6f893f8bd05331b4330b7d57f298"
server_key_iv: "7d537f3f9a3a598d1fbe9340d36264d5"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.030059 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "685315caecd64ef890348ab3ad59753c"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "c93f6f893f8bd05331b4330b7d57f298"
server_key_iv: "7d537f3f9a3a598d1fbe9340d36264d5"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.031313 8482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:43.032130 8497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.032328 8482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:43.032434 8482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "685315caecd64ef890348ab3ad59753c"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "c93f6f893f8bd05331b4330b7d57f298"
server_key_iv: "7d537f3f9a3a598d1fbe9340d36264d5"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.032506 8482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:43.053485 8482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:43.055858 8482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:43.056025 8482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:43.056264 8482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:43.056581 8482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:43.056632 8482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.056667 8482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:43.056694 8482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.062604 8482 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:40527
I20260619 02:40:43.062676 8610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:40527 every 8 connection(s)
I20260619 02:40:43.062988 8482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:43.067756 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8482
I20260619 02:40:43.067867 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:43.068042 7716 external_mini_cluster.cc:1428] Setting key e31545a315a1fa791b9e1921577dd8b2
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:43.073357 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.063740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:38447 (local address 127.7.137.62:34917)
0619 02:40:43.063826 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.063828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.064033 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:40:43.065029 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.065030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.065041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.065104 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.065697 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.066200 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.067227 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.067636 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.067705 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.070390 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.070392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.070392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.070400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.071873 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.072297 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.072298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.072299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.072370 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.072621 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.072622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.072622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.072791 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.072879 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.073164 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.073260 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:40:43.073558 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.063730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34917 (local address 127.7.137.2:38447)
0619 02:40:43.063905 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.063941 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.064917 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.065167 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.065170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.065419 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:43.065610 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.065614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.066300 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.066301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.067129 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.067131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.067743 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.067745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.067937 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.068643 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:43.068681 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:43.070274 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:43.071974 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:43.071981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:43.071984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:43.072208 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:43.072437 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:43.072438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:43.072439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:43.072553 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:43.072881 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:43.072888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:43.073028 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.073130 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.073288 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
I20260619 02:40:43.073947 8611 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34917
I20260619 02:40:43.074060 8611 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:43.074241 8611 heartbeater.cc:511] Master 127.7.137.62:34917 requested a full tablet report, sending...
I20260619 02:40:43.074637 8286 ts_manager.cc:194] Registered new tserver with Master: 685315caecd64ef890348ab3ad59753c (127.7.137.2:40527)
I20260619 02:40:43.075023 8286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:38447
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:43.093597 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:43.173370 8616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:43.173576 8616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:43.173599 8616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:43.174736 8616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:43.174777 8616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:43.174803 8616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:43.176244 8616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:43.176476 8616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:43.176738 8616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:43.181629 8624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:43.181615 8621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:43.181617 8622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:43.181807 8616 server_base.cc:1034] running on GCE node
I20260619 02:40:43.182160 8616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:43.182418 8616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:43.183602 8616 hybrid_clock.cc:648] HybridClock initialized: now 1781836843183585 us; error 29 us; skew 500 ppm
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:43.186167 8616 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:43.186655 8616 webserver.cc:460] Webserver started at http://127.7.137.3:39185/ using document root <none> and password file <none>
I20260619 02:40:43.186900 8616 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:43.186972 8616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:43.187131 8616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:43.188239 8616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "ca63e8b4ade7472aa5cb7c84128f1721"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "9f34ad2619af709c29704b09e4771132"
server_key_iv: "42c779a898ab775d5107fbe729174bc6"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.188632 8616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "ca63e8b4ade7472aa5cb7c84128f1721"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "9f34ad2619af709c29704b09e4771132"
server_key_iv: "42c779a898ab775d5107fbe729174bc6"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.190052 8616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:43.190835 8631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.191004 8616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:43.191107 8616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "ca63e8b4ade7472aa5cb7c84128f1721"
format_stamp: "Formatted at 2026-06-19 02:40:43 on dist-test-slave-pdnb"
server_key: "9f34ad2619af709c29704b09e4771132"
server_key_iv: "42c779a898ab775d5107fbe729174bc6"
server_key_version: "encryptionkey@0"
I20260619 02:40:43.191195 8616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:43.217034 8616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:43.219291 8616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:43.219484 8616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:43.219794 8616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:43.220140 8616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:43.220202 8616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.220248 8616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:43.220288 8616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:43.226599 8616 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:42351
I20260619 02:40:43.226639 8744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:42351 every 8 connection(s)
I20260619 02:40:43.226997 8616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:43.228632 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8616
I20260619 02:40:43.228710 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:43.228911 7716 external_mini_cluster.cc:1428] Setting key b51e870c33855ab6035a6123ce5d3b18
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:43.236990 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.227739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:43043 (local address 127.7.137.62:34917)
0619 02:40:43.227895 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.227898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.228089 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:40:43.229311 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.229311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.229324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.229374 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.230112 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.230636 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.231870 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.232073 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.232133 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.234370 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.234371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.234371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.234377 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.235675 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.236099 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.236119 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.236120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.236180 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.236413 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.236414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.236414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.236509 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.236557 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.236864 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.236928 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260619 02:40:43.237251 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.227706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34917 (local address 127.7.137.3:43043)
0619 02:40:43.227918 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.227988 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.229155 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.229448 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.229452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.229809 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:43.230012 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.230016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.230730 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.230731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.231755 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.231757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.232159 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.232160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.232315 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.232904 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:43.232914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:43.234282 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:43.235785 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:43.235790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:43.235792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:43.236016 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:43.236239 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:43.236240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:43.236240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:43.236367 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:43.236564 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:43.236568 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:43.236687 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.236849 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.236985 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260619 02:40:43.237780 8745 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34917
I20260619 02:40:43.237869 8745 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:43.238052 8745 heartbeater.cc:511] Master 127.7.137.62:34917 requested a full tablet report, sending...
I20260619 02:40:43.238430 8286 ts_manager.cc:194] Registered new tserver with Master: ca63e8b4ade7472aa5cb7c84128f1721 (127.7.137.3:42351)
I20260619 02:40:43.238776 8286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:43043
I20260619 02:40:43.240641 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:40:43.247731 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.242329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.7.137.62:34917)
0619 02:40:43.242410 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.242414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.242508 (+ 94us) server_negotiation.cc:407] Connection header received
0619 02:40:43.242559 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.242560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.242571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.242637 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.243138 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.243599 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.244261 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.244446 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.244494 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.245680 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.245681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.245681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.245686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.246815 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.247109 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.247110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.247110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.247155 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.247309 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.247310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.247310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.247447 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.247521 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.247621 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.247673 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:40:43.249768 8286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:43.251325 8286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:43.257767 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.7.137.2:40527)
0619 02:40:43.254460 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.254465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.254474 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:43.254508 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.254510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.254523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.254592 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:43.254709 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255691 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.256611 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257347 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257437 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257497 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.257641 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.257690 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":100,"threads_started":1}
I20260619 02:40:43.257778 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:40527 (local address 127.0.0.1:47930)
0619 02:40:43.254249 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.254260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.254301 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.254605 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.254605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.254607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:43.254650 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.254652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255783 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.255784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.256518 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.256520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257445 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.257530 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257545 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.257616 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.257668 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":34,"threads_started":1}
I20260619 02:40:43.257951 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.7.137.1:39157)
0619 02:40:43.253736 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.253742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.253847 (+ 105us) server_negotiation.cc:407] Connection header received
0619 02:40:43.253944 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.253947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.253961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.254038 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:43.254331 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255177 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.256859 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257466 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257534 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257587 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.257819 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.257862 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260619 02:40:43.258216 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.7.137.3:42351)
0619 02:40:43.253973 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.253977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.253986 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:43.254044 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.254047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.254061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.254144 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:43.254279 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255475 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.256747 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257415 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257471 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257516 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.257817 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.258137 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":71,"threads_started":1}
I20260619 02:40:43.258358 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42351 (local address 127.0.0.1:33568)
0619 02:40:43.253732 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.253771 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.253836 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.254144 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.254145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.254147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:43.254212 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.254214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255580 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.255580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.256646 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.256648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257556 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.257636 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257639 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.257692 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.257730 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":68,"threads_started":1}
I20260619 02:40:43.258700 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.253450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:39157 (local address 127.0.0.1:51278)
0619 02:40:43.253732 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.253771 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.253836 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.254060 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.254063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.254065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:43.254134 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.254136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.255273 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.255274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.256774 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.256776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.257619 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.257620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.257683 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.257685 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.257731 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.257768 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":36,"thread_start_us":53,"threads_started":1}
I20260619 02:40:43.258730 8545 tablet_service.cc:1467] Processing CreateTablet for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7 (DEFAULT_TABLE table=test-table [id=f3f07003a7d443b9b3fc9e99cb2d4cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:43.259011 8679 tablet_service.cc:1467] Processing CreateTablet for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7 (DEFAULT_TABLE table=test-table [id=f3f07003a7d443b9b3fc9e99cb2d4cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:43.259123 8545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:43.259333 8679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:43.259311 8411 tablet_service.cc:1467] Processing CreateTablet for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7 (DEFAULT_TABLE table=test-table [id=f3f07003a7d443b9b3fc9e99cb2d4cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:43.259601 8411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:43.262070 8759 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: Bootstrap starting.
I20260619 02:40:43.262629 8760 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Bootstrap starting.
I20260619 02:40:43.262717 8759 tablet_bootstrap.cc:654] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:43.263314 8760 tablet_bootstrap.cc:654] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:43.263316 8759 log.cc:824] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:43.263880 8760 log.cc:824] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:43.264086 8759 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: No bootstrap required, opened a new log
I20260619 02:40:43.264159 8759 ts_tablet_manager.cc:1397] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:43.264207 8761 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: Bootstrap starting.
I20260619 02:40:43.264583 8760 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: No bootstrap required, opened a new log
I20260619 02:40:43.264667 8760 ts_tablet_manager.cc:1397] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:43.264853 8761 tablet_bootstrap.cc:654] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:43.265825 8761 log.cc:824] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:43.266343 8760 raft_consensus.cc:348] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.266520 8760 raft_consensus.cc:374] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:43.266562 8760 raft_consensus.cc:729] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 685315caecd64ef890348ab3ad59753c, State: Initialized, Role: FOLLOWER
I20260619 02:40:43.266623 8761 tablet_bootstrap.cc:492] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: No bootstrap required, opened a new log
I20260619 02:40:43.266690 8761 ts_tablet_manager.cc:1397] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:40:43.266693 8760 consensus_queue.cc:260] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.266837 8759 raft_consensus.cc:348] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.266979 8611 heartbeater.cc:503] Master 127.7.137.62:34917 was elected leader, sending a full tablet report...
I20260619 02:40:43.267015 8759 raft_consensus.cc:374] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:43.267056 8759 raft_consensus.cc:729] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca63e8b4ade7472aa5cb7c84128f1721, State: Initialized, Role: FOLLOWER
I20260619 02:40:43.266971 8760 ts_tablet_manager.cc:1428] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:43.267171 8759 consensus_queue.cc:260] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.267432 8759 ts_tablet_manager.cc:1428] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:40:43.267565 8745 heartbeater.cc:503] Master 127.7.137.62:34917 was elected leader, sending a full tablet report...
I20260619 02:40:43.268631 8761 raft_consensus.cc:348] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.268832 8761 raft_consensus.cc:374] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:43.268860 8761 raft_consensus.cc:729] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d5353d6e6d45d3bcdd6fa2fd41697f, State: Initialized, Role: FOLLOWER
I20260619 02:40:43.268986 8761 consensus_queue.cc:260] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.269244 8761 ts_tablet_manager.cc:1428] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:40:43.269398 8477 heartbeater.cc:503] Master 127.7.137.62:34917 was elected leader, sending a full tablet report...
I20260619 02:40:43.304544 8767 raft_consensus.cc:482] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:43.304665 8767 raft_consensus.cc:504] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.305058 8767 leader_election.cc:288] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca63e8b4ade7472aa5cb7c84128f1721 (127.7.137.3:42351), 685315caecd64ef890348ab3ad59753c (127.7.137.2:40527)
I20260619 02:40:43.307911 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.305314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:40493 (local address 127.7.137.2:40527)
0619 02:40:43.305358 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.305360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.305484 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:40:43.305545 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.305546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.305558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.305630 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:43.305800 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.306249 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307177 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.307548 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307635 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.307648 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.307801 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.307836 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:40:43.307919 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.305225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:40527 (local address 127.7.137.1:40493)
0619 02:40:43.305414 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.305423 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.305493 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.305634 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.305635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.305636 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:43.305710 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.305711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.306337 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.306338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.307084 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.307085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.307637 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.307707 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.307711 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.307793 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.307832 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260619 02:40:43.308084 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.305172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42351 (local address 127.7.137.1:39773)
0619 02:40:43.305216 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:43.305229 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:43.305284 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:43.305568 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:43.305569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:43.305570 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:43.305641 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.305642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.306269 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.306269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.307029 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:43.307031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.307790 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:43.307882 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.307885 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:43.307960 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:40:43.308009 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":2}
I20260619 02:40:43.308082 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.305369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:39773 (local address 127.7.137.3:42351)
0619 02:40:43.305417 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.305419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.305436 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:40:43.305459 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.305460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.305468 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.305555 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:43.305713 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.306186 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307236 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.307692 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.307790 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.307800 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.307965 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.308005 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:40:43.308435 8565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22735a5e08c4bdfbb00ebdeaf9be8f7" candidate_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685315caecd64ef890348ab3ad59753c" is_pre_election: true
I20260619 02:40:43.308434 8699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22735a5e08c4bdfbb00ebdeaf9be8f7" candidate_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" is_pre_election: true
I20260619 02:40:43.308591 8565 raft_consensus.cc:2454] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d5353d6e6d45d3bcdd6fa2fd41697f in term 0.
I20260619 02:40:43.308591 8699 raft_consensus.cc:2454] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d5353d6e6d45d3bcdd6fa2fd41697f in term 0.
I20260619 02:40:43.308823 8364 leader_election.cc:302] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 685315caecd64ef890348ab3ad59753c, b1d5353d6e6d45d3bcdd6fa2fd41697f; no voters:
I20260619 02:40:43.308980 8767 raft_consensus.cc:2781] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:43.309042 8767 raft_consensus.cc:482] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:43.309067 8767 raft_consensus.cc:3037] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:43.309762 8767 raft_consensus.cc:504] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.309918 8767 leader_election.cc:288] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [CANDIDATE]: Term 1 election: Requested vote from peers ca63e8b4ade7472aa5cb7c84128f1721 (127.7.137.3:42351), 685315caecd64ef890348ab3ad59753c (127.7.137.2:40527)
I20260619 02:40:43.310101 8565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22735a5e08c4bdfbb00ebdeaf9be8f7" candidate_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685315caecd64ef890348ab3ad59753c"
I20260619 02:40:43.310102 8699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22735a5e08c4bdfbb00ebdeaf9be8f7" candidate_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca63e8b4ade7472aa5cb7c84128f1721"
I20260619 02:40:43.310195 8699 raft_consensus.cc:3037] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:43.310199 8565 raft_consensus.cc:3037] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:43.311054 8565 raft_consensus.cc:2454] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d5353d6e6d45d3bcdd6fa2fd41697f in term 1.
I20260619 02:40:43.311079 8699 raft_consensus.cc:2454] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P ca63e8b4ade7472aa5cb7c84128f1721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d5353d6e6d45d3bcdd6fa2fd41697f in term 1.
I20260619 02:40:43.311244 8364 leader_election.cc:302] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 685315caecd64ef890348ab3ad59753c, b1d5353d6e6d45d3bcdd6fa2fd41697f; no voters:
I20260619 02:40:43.311350 8767 raft_consensus.cc:2781] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:43.311520 8767 raft_consensus.cc:686] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [term 1 LEADER]: Becoming Leader. State: Replica: b1d5353d6e6d45d3bcdd6fa2fd41697f, State: Running, Role: LEADER
I20260619 02:40:43.311628 8767 consensus_queue.cc:237] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } }
I20260619 02:40:43.312463 8285 catalog_manager.cc:5495] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P b1d5353d6e6d45d3bcdd6fa2fd41697f reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d5353d6e6d45d3bcdd6fa2fd41697f (127.7.137.1). New cstate: current_term: 1 leader_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63e8b4ade7472aa5cb7c84128f1721" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d5353d6e6d45d3bcdd6fa2fd41697f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 39157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "685315caecd64ef890348ab3ad59753c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 40527 } health_report { overall_health: UNKNOWN } } }
W20260619 02:40:43.313508 8612 tablet.cc:2369] T e22735a5e08c4bdfbb00ebdeaf9be8f7 P 685315caecd64ef890348ab3ad59753c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:43.334765 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.326807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.7.137.1:39157)
0619 02:40:43.326887 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.326889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.326970 (+ 81us) server_negotiation.cc:407] Connection header received
0619 02:40:43.327013 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.327014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.327025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.327085 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.327626 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.328178 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.328954 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.329140 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.329204 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.331907 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.331917 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.331918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.331925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.333867 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.334160 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.334162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.334163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.334214 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.334353 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.334354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.334354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.334478 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.334583 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.334640 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.334698 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:43.357596 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.350618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.7.137.1:39157)
0619 02:40:43.350706 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.350708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.350807 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:40:43.350899 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.350900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.350916 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.350982 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.351535 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.352031 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.352776 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.352986 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.353072 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.355202 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.355203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.355204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.355211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.356406 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.356763 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.356764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.356765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.356832 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.357031 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.357033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.357033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.357131 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.357230 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.357420 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.357494 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:43.379176 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:43.372307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.7.137.1:39157)
0619 02:40:43.372369 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:43.372371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:43.372470 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:40:43.372521 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:43.372522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:43.372531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:43.372585 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:43.373144 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.373635 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.374329 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:43.374533 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:43.374588 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:43.376648 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:43.376649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:43.376650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:43.376658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:43.377937 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.378440 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.378441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.378446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.378493 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:43.378689 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:43.378690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:43.378690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:43.378787 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:43.378887 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:43.378997 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:40:43.379073 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 02:40:43.379642 8391 tablet_service.cc:2908] Rejecting scan request for tablet e22735a5e08c4bdfbb00ebdeaf9be8f7: Uninitialized: clean time has not yet been initialized
I20260619 02:40:43.380481 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8348
I20260619 02:40:43.385643 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8482
I20260619 02:40:43.389744 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8616
I20260619 02:40:43.393837 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8254
2026-06-19T02:40:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1953 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8783](info): set up 2 sockets
Jun 19 02:40:43 dist-test-slave-pdnb krb5kdc[8783](info): commencing operation
krb5kdc: starting...
W20260619 02:40:44.556943 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260619 02:40:44.884472 7716 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.327s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260619 02:40:45.073093 7716 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.189s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260619 02:40:45.378575 7716 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.305s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/test-user.keytab.
W20260619 02:40:46.388579 7716 mini_kdc.cc:302] Time spent creating keytab for test-user: real 1.010s user 0.000s sys 0.001s
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:46.429725 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36949
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:46.508739 8796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:46.508971 8796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:46.509008 8796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:46.510165 8796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:46.510246 8796 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:40:46.510272 8796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:46.510304 8796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:46.510329 8796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:46.510361 8796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:46.511822 8796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36949
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:46.512095 8796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:46.512323 8796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:46.517264 8801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.517349 8796 server_base.cc:1034] running on GCE node
W20260619 02:40:46.517328 8802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:46.517277 8804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.517745 8796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:46.517989 8796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:46.519171 8796 hybrid_clock.cc:648] HybridClock initialized: now 1781836846519152 us; error 39 us; skew 500 ppm
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:46.524887 8796 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:46.525355 8796 webserver.cc:460] Webserver started at http://127.7.137.62:32805/ using document root <none> and password file <none>
I20260619 02:40:46.525591 8796 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:46.525660 8796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:46.525796 8796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:46.526832 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "9e0754151dee469dbe1fae88c017a578"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5c759ce3b8b3956837c94cb73873fdd2"
server_key_iv: "17672892785dff4e918a501a23d65590"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.527199 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "9e0754151dee469dbe1fae88c017a578"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5c759ce3b8b3956837c94cb73873fdd2"
server_key_iv: "17672892785dff4e918a501a23d65590"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.528525 8796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:46.529361 8811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.529666 8796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:46.529778 8796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "9e0754151dee469dbe1fae88c017a578"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5c759ce3b8b3956837c94cb73873fdd2"
server_key_iv: "17672892785dff4e918a501a23d65590"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.529867 8796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:46.539819 8796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:46.541770 8796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:46.541929 8796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:46.546538 8796 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36949
I20260619 02:40:46.546578 8873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36949 every 8 connection(s)
I20260619 02:40:46.546887 8796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:46.547458 8874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:46.549731 8874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Bootstrap starting.
I20260619 02:40:46.550305 8874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:46.550590 8874 log.cc:824] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:46.551215 8874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: No bootstrap required, opened a new log
I20260619 02:40:46.553332 8874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } }
I20260619 02:40:46.553488 8874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:46.553535 8874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0754151dee469dbe1fae88c017a578, State: Initialized, Role: FOLLOWER
I20260619 02:40:46.553680 8874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } }
I20260619 02:40:46.553759 8874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:46.553802 8874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:46.553874 8874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:46.554147 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8796
I20260619 02:40:46.554220 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:46.554389 7716 external_mini_cluster.cc:1428] Setting key 765fb6c99299bf421de3669d1259d7f8
I20260619 02:40:46.554508 8874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } }
I20260619 02:40:46.554625 8874 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0754151dee469dbe1fae88c017a578; no voters:
I20260619 02:40:46.554778 8874 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:46.554805 8877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:46.555012 8877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [term 1 LEADER]: Becoming Leader. State: Replica: 9e0754151dee469dbe1fae88c017a578, State: Running, Role: LEADER
I20260619 02:40:46.555043 8874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:46.555195 8877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } }
I20260619 02:40:46.555653 8877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0754151dee469dbe1fae88c017a578. Latest consensus state: current_term: 1 leader_uuid: "9e0754151dee469dbe1fae88c017a578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } } }
I20260619 02:40:46.555770 8877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:46.556171 8880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0754151dee469dbe1fae88c017a578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0754151dee469dbe1fae88c017a578" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36949 } } }
I20260619 02:40:46.556245 8880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:46.556360 8886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:46.556912 8886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:46.558440 8886 catalog_manager.cc:1347] Generated new cluster ID: 648bd49dd7b744b78d21fd8d2707564f
I20260619 02:40:46.558495 8886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:46.565126 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.554923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.7.137.62:36949)
0619 02:40:46.555109 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:46.555117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:46.555351 (+ 234us) server_negotiation.cc:407] Connection header received
0619 02:40:46.555469 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:46.555476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:46.555530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:46.555673 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:46.556556 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.558024 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.558923 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.559480 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.559617 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.562027 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:46.562031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:46.562031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:46.562042 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:46.563649 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.564025 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.564027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.564028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.564119 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.564327 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.564327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.564328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.564550 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:46.564726 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:46.564835 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:40:46.564978 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260619 02:40:46.566107 8886 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:46.566584 8886 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:46.570036 8886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Generated new TSK 0
I20260619 02:40:46.570233 8886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:46.595482 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:46.676651 8900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:46.676898 8900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:46.676921 8900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:46.678125 8900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:46.678170 8900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:46.678191 8900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:46.678208 8900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:46.679771 8900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:46.680018 8900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:46.680261 8900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:46.685117 8905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.685259 8900 server_base.cc:1034] running on GCE node
W20260619 02:40:46.685114 8908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:46.685391 8906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.685627 8900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:46.685878 8900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:46.687063 8900 hybrid_clock.cc:648] HybridClock initialized: now 1781836846687044 us; error 45 us; skew 500 ppm
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:46.689860 8900 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:46.690310 8900 webserver.cc:460] Webserver started at http://127.7.137.1:45323/ using document root <none> and password file <none>
I20260619 02:40:46.690507 8900 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:46.690546 8900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:46.690642 8900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:46.691658 8900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "8dde625c6fd445f8853e5e961ea3ab6f"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5b98bc7462afcdd4e30f0d75275f9d06"
server_key_iv: "7d5cb0b4ab2dc6379973b2447e144774"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.691969 8900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "8dde625c6fd445f8853e5e961ea3ab6f"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5b98bc7462afcdd4e30f0d75275f9d06"
server_key_iv: "7d5cb0b4ab2dc6379973b2447e144774"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.693392 8900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:46.694109 8915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.694319 8900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:46.694406 8900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "8dde625c6fd445f8853e5e961ea3ab6f"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "5b98bc7462afcdd4e30f0d75275f9d06"
server_key_iv: "7d5cb0b4ab2dc6379973b2447e144774"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.694461 8900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:46.721489 8900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:46.723588 8900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:46.723716 8900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:46.724241 8900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:46.724318 8900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.724365 8900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:46.724393 8900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.731005 8900 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:42079
I20260619 02:40:46.731091 9030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:42079 every 8 connection(s)
I20260619 02:40:46.731386 8900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:46.735145 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.724792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:56705 (local address 127.7.137.62:36949)
0619 02:40:46.724868 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:46.724872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:46.725125 (+ 253us) server_negotiation.cc:407] Connection header received
0619 02:40:46.725760 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:46.725762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:46.725778 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:46.725857 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:46.726824 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.727691 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.729308 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.729709 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.729795 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.732389 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:46.732390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:46.732391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:46.732397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:46.733975 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.734356 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.734357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.734358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.734408 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.734608 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.734608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.734609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.734706 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:46.734763 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:46.735018 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:40:46.735081 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:40:46.735347 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.724769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.1:56705)
0619 02:40:46.724984 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:46.725028 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:46.725642 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:46.725963 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:46.725968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:46.726370 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:46.726690 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.726696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.727804 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.727805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.729220 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.729223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.729803 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.729804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.729959 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.730532 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:46.730543 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:46.732301 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:46.734045 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.734051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.734053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.734268 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.734464 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.734465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.734465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.734556 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.734756 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:46.734760 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:46.734906 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 02:40:46.734991 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:40:46.735107 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":145,"threads_started":1}
I20260619 02:40:46.735924 9031 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36949
I20260619 02:40:46.736019 9031 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:46.736199 9031 heartbeater.cc:511] Master 127.7.137.62:36949 requested a full tablet report, sending...
I20260619 02:40:46.736681 8828 ts_manager.cc:194] Registered new tserver with Master: 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1:42079)
I20260619 02:40:46.737414 8828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:56705
I20260619 02:40:46.740620 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 8900
I20260619 02:40:46.740756 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:46.740945 7716 external_mini_cluster.cc:1428] Setting key 71b2965e4885e7fec925275f0d75b72c
I20260619 02:40:46.743041 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.736947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.1:39781)
0619 02:40:46.737076 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:46.737087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:46.737145 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:46.737256 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:46.737256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:46.737438 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:46.737487 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.737489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.738207 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.738208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.738673 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.738674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.739073 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.739074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.739135 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.739604 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:46.739609 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:46.739903 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:46.741876 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.741877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.741878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.742041 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.742271 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.742272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.742272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.742336 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.742630 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:46.742631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:46.742679 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:40:46.742753 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:40:46.742907 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260619 02:40:46.743050 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.737025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:39781 (local address 127.7.137.62:36949)
0619 02:40:46.737070 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:46.737072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:46.737130 (+ 58us) server_negotiation.cc:407] Connection header received
0619 02:40:46.737201 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:46.737202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:46.737210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:46.737254 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:46.737569 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.738107 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.738766 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.738995 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.739064 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.739986 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:46.739987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:46.739987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:46.739993 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:46.741747 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.742116 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.742117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.742117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.742180 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.742407 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.742408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.742408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.742558 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:46.742629 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:46.742794 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:40:46.742900 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:46.771432 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:46.851864 9040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:46.852064 9040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:46.852088 9040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:46.853319 9040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:46.853360 9040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:46.853380 9040 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:46.853399 9040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:46.854859 9040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:46.855082 9040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:46.855311 9040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:46.860267 9046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:46.860351 9048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.860340 9040 server_base.cc:1034] running on GCE node
W20260619 02:40:46.860334 9045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:46.860697 9040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:46.860957 9040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:46.862097 9040 hybrid_clock.cc:648] HybridClock initialized: now 1781836846862065 us; error 48 us; skew 500 ppm
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:46.864518 9040 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:46.865000 9040 webserver.cc:460] Webserver started at http://127.7.137.2:40849/ using document root <none> and password file <none>
I20260619 02:40:46.865243 9040 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:46.865312 9040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:46.865463 9040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:46.866509 9040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "c1d8edba14b5426ce40514d91be69ae4"
server_key_iv: "1a803ebf4758e4297fff3e74da3102e6"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.866878 9040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "c1d8edba14b5426ce40514d91be69ae4"
server_key_iv: "1a803ebf4758e4297fff3e74da3102e6"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.868157 9040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:46.868912 9055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.869115 9040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:46.869231 9040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
format_stamp: "Formatted at 2026-06-19 02:40:46 on dist-test-slave-pdnb"
server_key: "c1d8edba14b5426ce40514d91be69ae4"
server_key_iv: "1a803ebf4758e4297fff3e74da3102e6"
server_key_version: "encryptionkey@0"
I20260619 02:40:46.869351 9040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:46.890425 9040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:46.892426 9040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:46.892601 9040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:46.893206 9040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:46.893292 9040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:46.893344 9040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:46.893376 9040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:40:46 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:46.901065 9040 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:36661
I20260619 02:40:46.901144 9170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:36661 every 8 connection(s)
I20260619 02:40:46.901541 9040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:46.903587 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.893734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:60331 (local address 127.7.137.62:36949)
0619 02:40:46.893806 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:46.893808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:46.894018 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:40:46.894504 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:46.894505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:46.894520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:46.894598 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:46.895495 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.896442 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.897689 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.897893 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.897972 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.900766 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:46.900768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:46.900768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:46.900777 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:46.902479 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.902832 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.902833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.902833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.902889 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.903092 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.903092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.903093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.903198 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:46.903250 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:46.903460 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 02:40:46.903520 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:40:46.903733 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.893716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.2:60331)
0619 02:40:46.893882 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:46.893922 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:46.894409 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:46.894694 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:46.894699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:46.895055 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:46.895381 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.895388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.896549 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.896551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.897587 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.897589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.897989 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.897990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.898140 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.898705 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:46.898718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:46.900623 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:46.902551 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.902555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.902557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.902764 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.902945 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.902946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.902946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.903036 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.903251 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:46.903255 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:46.903366 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 02:40:46.903444 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:40:46.903553 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260619 02:40:46.904250 9171 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36949
I20260619 02:40:46.904363 9171 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:46.904543 9171 heartbeater.cc:511] Master 127.7.137.62:36949 requested a full tablet report, sending...
I20260619 02:40:46.904974 8828 ts_manager.cc:194] Registered new tserver with Master: d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661)
I20260619 02:40:46.905754 8828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:60331
I20260619 02:40:46.906395 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9040
I20260619 02:40:46.906481 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:46.906646 7716 external_mini_cluster.cc:1428] Setting key ebf2c7903e9f6846ce2f3ef331ccb0ce
I20260619 02:40:46.912111 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.905263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.2:33609)
0619 02:40:46.905388 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:46.905398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:46.905447 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:46.905670 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:46.905670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:46.905883 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:46.905959 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.905962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.906807 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.906808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.907630 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:46.907633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.907953 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.907954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:46.908036 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.908429 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:46.908434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:46.908853 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:46.910916 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.910917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.910917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.911077 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.911310 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:46.911311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:46.911311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:46.911517 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:46.911890 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:46.911890 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:46.911925 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:40:46.911973 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:40:46.912048 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260619 02:40:46.912261 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:46.905503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:33609 (local address 127.7.137.62:36949)
0619 02:40:46.905572 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:46.905574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:46.905580 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:46.905595 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:46.905595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:46.905602 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:46.905665 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:46.906018 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.906695 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.907705 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:46.907882 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:46.907936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:46.908947 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:46.908949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:46.908949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:46.908957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:46.910829 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.911155 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.911156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.911157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.911239 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:46.911596 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:46.911597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:46.911598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:46.911811 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:46.911890 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:46.912005 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:40:46.912091 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:46.933736 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:47.014953 9180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:47.015146 9180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:47.015168 9180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:47.016322 9180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:47.016363 9180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:47.016383 9180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:47.016446 9180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:47.017869 9180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:47.018113 9180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:47.018352 9180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:47.023281 9185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:47.023343 9188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:47.023293 9186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:47.023540 9180 server_base.cc:1034] running on GCE node
I20260619 02:40:47.023756 9180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:47.024014 9180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:47.025171 9180 hybrid_clock.cc:648] HybridClock initialized: now 1781836847025154 us; error 35 us; skew 500 ppm
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836847, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:47.027671 9180 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:47.028134 9180 webserver.cc:460] Webserver started at http://127.7.137.3:35183/ using document root <none> and password file <none>
I20260619 02:40:47.028388 9180 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:47.028432 9180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:47.028571 9180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:47.029690 9180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "cfe55ce640824513accde19f6faaea8a"
format_stamp: "Formatted at 2026-06-19 02:40:47 on dist-test-slave-pdnb"
server_key: "d1b28c77ac02f43104108959dff98247"
server_key_iv: "78bbf751a115e669276987955faf8308"
server_key_version: "encryptionkey@0"
I20260619 02:40:47.030040 9180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "cfe55ce640824513accde19f6faaea8a"
format_stamp: "Formatted at 2026-06-19 02:40:47 on dist-test-slave-pdnb"
server_key: "d1b28c77ac02f43104108959dff98247"
server_key_iv: "78bbf751a115e669276987955faf8308"
server_key_version: "encryptionkey@0"
I20260619 02:40:47.031423 9180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:47.032428 9195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.032614 9180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:47.032697 9180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "cfe55ce640824513accde19f6faaea8a"
format_stamp: "Formatted at 2026-06-19 02:40:47 on dist-test-slave-pdnb"
server_key: "d1b28c77ac02f43104108959dff98247"
server_key_iv: "78bbf751a115e669276987955faf8308"
server_key_version: "encryptionkey@0"
I20260619 02:40:47.032795 9180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:47.044967 9180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:47.047143 9180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:47.047300 9180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:47.047780 9180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:47.047817 9180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.047838 9180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:47.047851 9180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.054448 9180 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:42477
I20260619 02:40:47.054519 9310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:42477 every 8 connection(s)
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836847, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:47.054842 9180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:47.058076 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.048217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:41081 (local address 127.7.137.62:36949)
0619 02:40:47.048287 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.048291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.048509 (+ 218us) server_negotiation.cc:407] Connection header received
0619 02:40:47.049059 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.049060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.049074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.049160 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.049984 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.050816 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.052350 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.052684 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.052772 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.055363 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.055364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.055365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.055373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.056816 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.057262 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.057263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.057263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.057328 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.057565 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.057565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.057566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.057666 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.057716 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.057951 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.058016 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:40:47.058264 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.048189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.3:41081)
0619 02:40:47.048383 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.048423 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.048962 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.049216 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.049220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.049606 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:47.049884 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.049890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.050914 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.050915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.052276 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.052279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.052790 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.052791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.052921 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.053413 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:47.053423 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:47.055242 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:47.056899 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.056904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.056906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.057188 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.057395 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.057396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.057397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.057509 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.057719 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:47.057723 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:47.057829 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.057924 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.058037 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":124,"threads_started":1}
I20260619 02:40:47.058604 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9180
I20260619 02:40:47.058701 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:47.058701 9311 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36949
I20260619 02:40:47.058779 9311 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:47.058894 7716 external_mini_cluster.cc:1428] Setting key fb98a65d8628de1b2e3aa373f5d3a86d
I20260619 02:40:47.058914 9311 heartbeater.cc:511] Master 127.7.137.62:36949 requested a full tablet report, sending...
I20260619 02:40:47.059365 8827 ts_manager.cc:194] Registered new tserver with Master: cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477)
I20260619 02:40:47.059866 8827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:41081
I20260619 02:40:47.060882 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:40:47.065797 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.059770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.7.137.3:53313)
0619 02:40:47.059890 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.059900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.059956 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.060075 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.060076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.060268 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:47.060341 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.060342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.061186 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.061187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.061697 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.061698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.062096 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.062097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.062178 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.062610 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:47.062615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:47.062887 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:47.064678 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.064679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.064679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.064947 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.065164 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.065165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.065165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.065239 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.065516 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:47.065516 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:47.065551 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.065605 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.065680 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":9,"thread_start_us":76,"threads_started":1}
I20260619 02:40:47.065822 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.059766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:53313 (local address 127.7.137.62:36949)
0619 02:40:47.059833 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.059835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.059961 (+ 126us) server_negotiation.cc:407] Connection header received
0619 02:40:47.060015 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.060016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.060025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.060073 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.060416 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.061105 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.061814 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.062013 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.062094 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.062990 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.062992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.062992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.062999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.064598 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.065020 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.065022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.065022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.065082 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.065315 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.065316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.065317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.065450 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.065522 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.065630 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.065700 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836847, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836847, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:47.083891 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.077376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.7.137.62:36949)
0619 02:40:47.077431 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.077432 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.077441 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:47.077467 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.077468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.077476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.077525 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.078034 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.078514 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.079203 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.079407 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.079461 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.081587 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.081588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.081589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.081594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.082966 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.083258 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.083259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.083259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.083331 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.083491 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.083491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.083491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.083583 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.083654 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.083747 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.083803 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:40:47.085475 8827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:47.087041 8827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:47.094099 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.7.137.2:36661)
0619 02:40:47.090071 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.090076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.090085 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:47.090120 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.090122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.090136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.090204 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:47.090422 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.091325 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.092824 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.093579 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.093653 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.093710 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.093954 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.093996 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":76,"threads_started":1}
I20260619 02:40:47.094105 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:36661 (local address 127.0.0.1:38402)
0619 02:40:47.089992 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.090000 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.090035 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.090279 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.090279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.090281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:47.090340 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.090341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.091405 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.091406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.092714 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.092764 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.093748 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.093749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.093834 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.093848 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.093926 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.093995 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":33,"thread_start_us":93,"threads_started":1}
I20260619 02:40:47.094105 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42079 (local address 127.0.0.1:54868)
0619 02:40:47.089818 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.089850 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.089911 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.090197 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.090198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.090199 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:47.090330 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.090332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.091615 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.091616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.092713 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.092748 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.093747 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.093748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.093836 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.093848 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.093925 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.093995 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":87,"thread_start_us":55,"threads_started":1}
I20260619 02:40:47.094660 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42477 (local address 127.0.0.1:39638)
0619 02:40:47.089799 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.089842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.089911 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.090179 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.090182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.090185 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:47.090259 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.090261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.092850 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.092850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.093513 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.093514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.094427 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.094428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.094499 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.094503 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.094549 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.094590 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":194,"threads_started":1}
I20260619 02:40:47.094707 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.7.137.1:42079)
0619 02:40:47.090042 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.090046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.090055 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:47.090090 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.090091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.090105 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.090242 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:47.090413 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.091521 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.092844 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.093579 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.093653 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.093710 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.094570 (+ 860us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.094608 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260619 02:40:47.094903 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.089579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.7.137.3:42477)
0619 02:40:47.089757 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.089763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.089931 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:40:47.089988 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.089991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.090009 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.090095 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:47.090381 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.091533 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.093601 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.094252 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.094330 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.094394 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.094772 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.094814 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260619 02:40:47.095423 9105 tablet_service.cc:1467] Processing CreateTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b (DEFAULT_TABLE table=test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.095441 8965 tablet_service.cc:1467] Processing CreateTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b (DEFAULT_TABLE table=test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.095434 9245 tablet_service.cc:1467] Processing CreateTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b (DEFAULT_TABLE table=test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.095762 9105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5d69adc54ef43598e33ce1a2d399e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.095762 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5d69adc54ef43598e33ce1a2d399e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.095762 9245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5d69adc54ef43598e33ce1a2d399e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.098667 9334 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Bootstrap starting.
I20260619 02:40:47.099023 9335 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Bootstrap starting.
I20260619 02:40:47.099293 9336 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Bootstrap starting.
I20260619 02:40:47.099679 9334 tablet_bootstrap.cc:654] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.099964 9335 tablet_bootstrap.cc:654] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.100018 9334 log.cc:824] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:47.100035 9336 tablet_bootstrap.cc:654] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.100296 9335 log.cc:824] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:47.100370 9336 log.cc:824] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:47.100844 9334 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: No bootstrap required, opened a new log
I20260619 02:40:47.100939 9334 ts_tablet_manager.cc:1397] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:47.101044 9336 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: No bootstrap required, opened a new log
I20260619 02:40:47.101055 9335 tablet_bootstrap.cc:492] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: No bootstrap required, opened a new log
I20260619 02:40:47.101114 9336 ts_tablet_manager.cc:1397] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:47.101114 9335 ts_tablet_manager.cc:1397] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:47.102309 9334 raft_consensus.cc:348] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.102468 9334 raft_consensus.cc:374] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.102511 9334 raft_consensus.cc:729] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c69039fce24e15a76b8b41f9e8cafc, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.102634 9334 consensus_queue.cc:260] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.102898 9334 ts_tablet_manager.cc:1428] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:47.103041 9171 heartbeater.cc:503] Master 127.7.137.62:36949 was elected leader, sending a full tablet report...
I20260619 02:40:47.103045 9335 raft_consensus.cc:348] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.103199 9335 raft_consensus.cc:374] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.103243 9335 raft_consensus.cc:729] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe55ce640824513accde19f6faaea8a, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.103216 9336 raft_consensus.cc:348] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.103364 9336 raft_consensus.cc:374] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.103351 9335 consensus_queue.cc:260] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.103404 9336 raft_consensus.cc:729] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.103523 9336 consensus_queue.cc:260] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.103626 9335 ts_tablet_manager.cc:1428] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:40:47.103734 9311 heartbeater.cc:503] Master 127.7.137.62:36949 was elected leader, sending a full tablet report...
I20260619 02:40:47.103750 9336 ts_tablet_manager.cc:1428] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:40:47.103899 9031 heartbeater.cc:503] Master 127.7.137.62:36949 was elected leader, sending a full tablet report...
W20260619 02:40:47.152194 9172 tablet.cc:2369] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:40:47.232183 9032 tablet.cc:2369] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:40:47.305397 9312 tablet.cc:2369] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:47.337414 9342 raft_consensus.cc:482] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:47.337551 9342 raft_consensus.cc:504] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.337977 9342 leader_election.cc:288] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661), cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477)
I20260619 02:40:47.341387 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.338190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42477 (local address 127.7.137.1:54365)
0619 02:40:47.338312 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.338326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.338387 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.338602 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.338602 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.338605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:47.338690 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.338692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.339354 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.339355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.340586 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.340589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.341128 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.341129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.341211 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.341216 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.341273 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.341315 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":8,"thread_start_us":68,"threads_started":1}
I20260619 02:40:47.341414 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.338387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:54365 (local address 127.7.137.3:42477)
0619 02:40:47.338448 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.338450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.338459 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:47.338485 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.338486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.338498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.338574 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:47.338769 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.339224 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.340667 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.341048 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.341104 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.341112 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.341299 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.341332 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:40:47.341562 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.338382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:32887 (local address 127.7.137.2:36661)
0619 02:40:47.338443 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.338445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.338453 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:47.338481 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.338482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.338500 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.338543 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:47.338708 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.339216 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.340711 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.341170 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.341252 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.341263 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.341439 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.341486 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:40:47.341791 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5d69adc54ef43598e33ce1a2d399e4b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a" is_pre_election: true
I20260619 02:40:47.341944 9265 raft_consensus.cc:2454] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.341950 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.338215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:36661 (local address 127.7.137.1:32887)
0619 02:40:47.338339 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.338351 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.338409 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.338578 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.338578 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.338581 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:47.338642 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.338643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.339301 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.339302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.340607 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.340609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.341295 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.341296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.341362 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.341366 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.341422 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.341869 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260619 02:40:47.342243 8919 leader_election.cc:302] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, cfe55ce640824513accde19f6faaea8a; no voters:
I20260619 02:40:47.342335 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5d69adc54ef43598e33ce1a2d399e4b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" is_pre_election: true
I20260619 02:40:47.342386 9342 raft_consensus.cc:2781] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:47.342428 9342 raft_consensus.cc:482] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:47.342458 9342 raft_consensus.cc:3037] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.342476 9125 raft_consensus.cc:2454] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.343197 9342 raft_consensus.cc:504] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.343335 9342 leader_election.cc:288] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Requested vote from peers d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661), cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477)
I20260619 02:40:47.343475 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5d69adc54ef43598e33ce1a2d399e4b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
I20260619 02:40:47.343509 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5d69adc54ef43598e33ce1a2d399e4b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a"
I20260619 02:40:47.343573 9125 raft_consensus.cc:3037] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.343570 9265 raft_consensus.cc:3037] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.344272 9125 raft_consensus.cc:2454] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.344297 9265 raft_consensus.cc:2454] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.344483 8917 leader_election.cc:302] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, d9c69039fce24e15a76b8b41f9e8cafc; no voters:
I20260619 02:40:47.344585 9342 raft_consensus.cc:2781] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:47.344751 9342 raft_consensus.cc:686] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 LEADER]: Becoming Leader. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Running, Role: LEADER
I20260619 02:40:47.344869 9342 consensus_queue.cc:237] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } }
I20260619 02:40:47.345688 8827 catalog_manager.cc:5495] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1). New cstate: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } health_report { overall_health: HEALTHY } } }
I20260619 02:40:47.379777 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.377068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54880 (local address 127.7.137.1:42079)
0619 02:40:47.377141 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.377143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.377154 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:47.377309 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.377310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.377332 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.377445 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:47.377598 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.378111 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.378905 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.379120 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.379197 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.379411 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:47.379524 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:47.379616 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.379660 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.379683 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:47.381659 9125 raft_consensus.cc:1261] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.381708 9265 raft_consensus.cc:1261] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.382052 9345 consensus_queue.cc:1035] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.382176 9342 consensus_queue.cc:1035] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.384544 9348 mvcc.cc:205] Tried to move back new op lower bound from 7298403726872522752 to 7298403726725251072. Current Snapshot: MvccSnapshot[applied={T|T < 7298403726872522752}]
I20260619 02:40:47.384763 9349 mvcc.cc:205] Tried to move back new op lower bound from 7298403726872522752 to 7298403726725251072. Current Snapshot: MvccSnapshot[applied={T|T < 7298403726872522752}]
I20260619 02:40:47.384864 9350 mvcc.cc:205] Tried to move back new op lower bound from 7298403726872522752 to 7298403726725251072. Current Snapshot: MvccSnapshot[applied={T|T < 7298403726872522752}]
I20260619 02:40:47.388017 8827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:47.388113 8827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:47.388882 8827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Sending DeleteTablet for 3 replicas of tablet b5d69adc54ef43598e33ce1a2d399e4b
I20260619 02:40:47.389205 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:54868
I20260619 02:40:47.389246 9245 tablet_service.cc:1514] Processing DeleteTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39638
I20260619 02:40:47.389237 9105 tablet_service.cc:1514] Processing DeleteTablet for tablet b5d69adc54ef43598e33ce1a2d399e4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:38402
I20260619 02:40:47.389437 9358 tablet_replica.cc:330] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: stopping tablet replica
I20260619 02:40:47.389528 9358 raft_consensus.cc:2229] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:47.389621 9358 raft_consensus.cc:2258] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.389664 8827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:47.389811 8827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:47.389948 9359 tablet_replica.cc:330] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: stopping tablet replica
I20260619 02:40:47.390049 9359 raft_consensus.cc:2229] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:47.390156 9359 raft_consensus.cc:2258] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.390525 9358 ts_tablet_manager.cc:1905] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.391065 8965 tablet_service.cc:1467] Processing CreateTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 (DEFAULT_TABLE table=test-table [id=75ddb7f6bbc24436b077871a2afe7f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.391079 9245 tablet_service.cc:1467] Processing CreateTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 (DEFAULT_TABLE table=test-table [id=75ddb7f6bbc24436b077871a2afe7f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.391198 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72d37b0d7474749bb26e5f0bf2744b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.391211 9245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72d37b0d7474749bb26e5f0bf2744b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.392163 9358 ts_tablet_manager.cc:1918] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:47.392237 9358 log.cc:1196] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/b5d69adc54ef43598e33ce1a2d399e4b
I20260619 02:40:47.392349 9335 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Bootstrap starting.
I20260619 02:40:47.392481 9358 ts_tablet_manager.cc:1939] T b5d69adc54ef43598e33ce1a2d399e4b P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting consensus metadata
I20260619 02:40:47.392501 9336 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Bootstrap starting.
I20260619 02:40:47.392865 8812 catalog_manager.cc:4841] TS 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1:42079): tablet b5d69adc54ef43598e33ce1a2d399e4b (table test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]) successfully deleted
I20260619 02:40:47.392943 9336 tablet_bootstrap.cc:654] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.393046 9105 tablet_service.cc:1467] Processing CreateTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 (DEFAULT_TABLE table=test-table [id=75ddb7f6bbc24436b077871a2afe7f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:47.393349 9359 ts_tablet_manager.cc:1905] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.393538 9105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72d37b0d7474749bb26e5f0bf2744b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.394505 9334 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Bootstrap starting.
I20260619 02:40:47.394695 9335 tablet_bootstrap.cc:654] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.394767 9359 ts_tablet_manager.cc:1918] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:47.394827 9359 log.cc:1196] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/b5d69adc54ef43598e33ce1a2d399e4b
I20260619 02:40:47.395073 9359 ts_tablet_manager.cc:1939] T b5d69adc54ef43598e33ce1a2d399e4b P cfe55ce640824513accde19f6faaea8a: Deleting consensus metadata
I20260619 02:40:47.395179 9334 tablet_bootstrap.cc:654] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.395390 9335 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: No bootstrap required, opened a new log
I20260619 02:40:47.395432 9335 ts_tablet_manager.cc:1397] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 02:40:47.395577 9335 raft_consensus.cc:348] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.395643 9335 raft_consensus.cc:374] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.395664 9335 raft_consensus.cc:729] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe55ce640824513accde19f6faaea8a, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.395711 9335 consensus_queue.cc:260] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.395798 9335 ts_tablet_manager.cc:1428] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.395833 9334 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: No bootstrap required, opened a new log
I20260619 02:40:47.395886 9334 ts_tablet_manager.cc:1397] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:47.396032 9334 raft_consensus.cc:348] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.396103 9334 raft_consensus.cc:374] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.396080 8815 catalog_manager.cc:4841] TS cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477): tablet b5d69adc54ef43598e33ce1a2d399e4b (table test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]) successfully deleted
I20260619 02:40:47.396124 9334 raft_consensus.cc:729] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c69039fce24e15a76b8b41f9e8cafc, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.396164 9334 consensus_queue.cc:260] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.396247 9334 ts_tablet_manager.cc:1428] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:47.396906 9336 tablet_bootstrap.cc:492] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: No bootstrap required, opened a new log
I20260619 02:40:47.396978 9336 ts_tablet_manager.cc:1397] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 02:40:47.397130 9336 raft_consensus.cc:348] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.397221 9336 raft_consensus.cc:374] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.397253 9336 raft_consensus.cc:729] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.397300 9336 consensus_queue.cc:260] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.397388 9336 ts_tablet_manager.cc:1428] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.400866 9360 tablet_replica.cc:330] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: stopping tablet replica
I20260619 02:40:47.401000 9360 raft_consensus.cc:2229] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:47.401135 9360 raft_consensus.cc:2258] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.401612 9360 ts_tablet_manager.cc:1905] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.402806 9360 ts_tablet_manager.cc:1918] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:40:47.402863 9360 log.cc:1196] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/b5d69adc54ef43598e33ce1a2d399e4b
I20260619 02:40:47.403090 9360 ts_tablet_manager.cc:1939] T b5d69adc54ef43598e33ce1a2d399e4b P d9c69039fce24e15a76b8b41f9e8cafc: Deleting consensus metadata
I20260619 02:40:47.403486 8813 catalog_manager.cc:4841] TS d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661): tablet b5d69adc54ef43598e33ce1a2d399e4b (table test-table [id=e3b892eb4da548f0943ce8b6d5cf50a1]) successfully deleted
I20260619 02:40:47.545601 9353 raft_consensus.cc:482] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:47.545699 9353 raft_consensus.cc:504] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.545853 9353 leader_election.cc:288] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661), cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477)
I20260619 02:40:47.546077 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72d37b0d7474749bb26e5f0bf2744b1" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" is_pre_election: true
I20260619 02:40:47.546077 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72d37b0d7474749bb26e5f0bf2744b1" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a" is_pre_election: true
I20260619 02:40:47.546188 9125 raft_consensus.cc:2454] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.546236 9265 raft_consensus.cc:2454] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.546350 8917 leader_election.cc:302] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, d9c69039fce24e15a76b8b41f9e8cafc; no voters:
I20260619 02:40:47.546473 9353 raft_consensus.cc:2781] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:47.546515 9353 raft_consensus.cc:482] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:47.546555 9353 raft_consensus.cc:3037] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.547237 9353 raft_consensus.cc:504] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.547399 9353 leader_election.cc:288] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Requested vote from peers d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661), cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477)
I20260619 02:40:47.547538 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72d37b0d7474749bb26e5f0bf2744b1" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
I20260619 02:40:47.547621 9125 raft_consensus.cc:3037] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.547636 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72d37b0d7474749bb26e5f0bf2744b1" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a"
I20260619 02:40:47.547709 9265 raft_consensus.cc:3037] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.548223 9125 raft_consensus.cc:2454] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.548442 8917 leader_election.cc:302] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, d9c69039fce24e15a76b8b41f9e8cafc; no voters:
I20260619 02:40:47.548453 9265 raft_consensus.cc:2454] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.548549 9353 raft_consensus.cc:2781] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:47.548594 9353 raft_consensus.cc:686] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 LEADER]: Becoming Leader. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Running, Role: LEADER
I20260619 02:40:47.548636 9353 consensus_queue.cc:237] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } }
I20260619 02:40:47.549225 8828 catalog_manager.cc:5495] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1). New cstate: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836846, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:47.608112 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.601373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36949 (local address 127.0.0.1:57912)
0619 02:40:47.601513 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.601525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.601605 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.601780 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.601781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.602070 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:47.602120 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.602122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.602751 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.602751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.603272 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.603273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.603605 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.603606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.603643 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.604195 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:47.604200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:47.605670 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:47.607121 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.607127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.607128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.607344 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.607542 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:47.607543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:47.607543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:47.607609 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:47.607832 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:47.607838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:47.607878 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.607936 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.608020 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260619 02:40:47.608208 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.601470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.7.137.62:36949)
0619 02:40:47.601587 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.601588 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.601598 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:47.601662 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.601663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.601672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.601732 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.602189 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.602688 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.603361 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.603543 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.603596 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.605745 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.605746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.605746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.605752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.607046 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.607409 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.607409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.607410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.607459 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.607668 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.607668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.607669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.607764 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.607813 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.607964 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.608067 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260619 02:40:47.610188 8828 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:57912:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:40:47.610419 8828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:47.611665 8965 tablet_service.cc:1467] Processing CreateTablet for tablet 1c930ea3ce684ee197d35e5a50c2312b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be992365c68643aa97592713e2632946]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:47.611677 9105 tablet_service.cc:1467] Processing CreateTablet for tablet 1c930ea3ce684ee197d35e5a50c2312b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be992365c68643aa97592713e2632946]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:47.611824 9105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c930ea3ce684ee197d35e5a50c2312b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.611824 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c930ea3ce684ee197d35e5a50c2312b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.612875 9245 tablet_service.cc:1467] Processing CreateTablet for tablet 1c930ea3ce684ee197d35e5a50c2312b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be992365c68643aa97592713e2632946]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:47.613044 9245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c930ea3ce684ee197d35e5a50c2312b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:47.613306 9336 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: Bootstrap starting.
I20260619 02:40:47.613384 9334 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: Bootstrap starting.
I20260619 02:40:47.613607 9336 tablet_bootstrap.cc:654] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.613700 9334 tablet_bootstrap.cc:654] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.614152 9336 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: No bootstrap required, opened a new log
I20260619 02:40:47.614198 9336 ts_tablet_manager.cc:1397] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:47.614342 9336 raft_consensus.cc:348] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.614419 9336 raft_consensus.cc:374] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.614439 9336 raft_consensus.cc:729] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.614486 9336 consensus_queue.cc:260] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.614573 9336 ts_tablet_manager.cc:1428] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.614647 9335 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: Bootstrap starting.
I20260619 02:40:47.614650 9345 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.614733 9345 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.615054 9335 tablet_bootstrap.cc:654] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:47.615290 9334 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: No bootstrap required, opened a new log
I20260619 02:40:47.615363 9334 ts_tablet_manager.cc:1397] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:47.615521 9334 raft_consensus.cc:348] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.615619 9334 raft_consensus.cc:374] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.615644 9334 raft_consensus.cc:729] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c69039fce24e15a76b8b41f9e8cafc, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.615687 9335 tablet_bootstrap.cc:492] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: No bootstrap required, opened a new log
I20260619 02:40:47.615693 9334 consensus_queue.cc:260] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.615736 9335 ts_tablet_manager.cc:1397] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:47.615803 9334 ts_tablet_manager.cc:1428] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.615864 9335 raft_consensus.cc:348] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.615936 9335 raft_consensus.cc:374] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:47.615872 9340 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.615957 9335 raft_consensus.cc:729] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe55ce640824513accde19f6faaea8a, State: Initialized, Role: FOLLOWER
I20260619 02:40:47.615969 9340 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.615999 9335 consensus_queue.cc:260] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.616097 9335 ts_tablet_manager.cc:1428] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:47.616120 9341 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.616197 9341 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.771232 9345 raft_consensus.cc:482] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:47.771353 9345 raft_consensus.cc:504] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.771549 9345 leader_election.cc:288] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477), d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661)
I20260619 02:40:47.771817 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c930ea3ce684ee197d35e5a50c2312b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a" is_pre_election: true
I20260619 02:40:47.771817 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c930ea3ce684ee197d35e5a50c2312b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" is_pre_election: true
I20260619 02:40:47.771920 9125 raft_consensus.cc:2454] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.771920 9265 raft_consensus.cc:2454] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 0.
I20260619 02:40:47.772083 8919 leader_election.cc:302] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, cfe55ce640824513accde19f6faaea8a; no voters:
I20260619 02:40:47.772210 9345 raft_consensus.cc:2781] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:47.772251 9345 raft_consensus.cc:482] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:47.772270 9345 raft_consensus.cc:3037] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.773052 9345 raft_consensus.cc:504] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.773216 9345 leader_election.cc:288] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Requested vote from peers cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477), d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661)
I20260619 02:40:47.773381 9265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c930ea3ce684ee197d35e5a50c2312b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe55ce640824513accde19f6faaea8a"
I20260619 02:40:47.773413 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c930ea3ce684ee197d35e5a50c2312b" candidate_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c69039fce24e15a76b8b41f9e8cafc"
I20260619 02:40:47.773458 9265 raft_consensus.cc:3037] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.773479 9125 raft_consensus.cc:3037] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:47.774178 9125 raft_consensus.cc:2454] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.774178 9265 raft_consensus.cc:2454] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dde625c6fd445f8853e5e961ea3ab6f in term 1.
I20260619 02:40:47.774356 8919 leader_election.cc:302] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dde625c6fd445f8853e5e961ea3ab6f, cfe55ce640824513accde19f6faaea8a; no voters:
I20260619 02:40:47.774503 9345 raft_consensus.cc:2781] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:47.774569 9345 raft_consensus.cc:686] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 LEADER]: Becoming Leader. State: Replica: 8dde625c6fd445f8853e5e961ea3ab6f, State: Running, Role: LEADER
I20260619 02:40:47.774603 9345 consensus_queue.cc:237] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } }
I20260619 02:40:47.774924 9353 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: TxnStatusTablet state changed. Reason: New leader 8dde625c6fd445f8853e5e961ea3ab6f. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.775020 9353 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:47.775199 9376 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:40:47.775303 9376 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:40:47.775286 8826 catalog_manager.cc:5495] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1). New cstate: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:47.797921 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.795426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54890 (local address 127.7.137.1:42079)
0619 02:40:47.795490 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.795492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.795500 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:47.795524 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.795525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.795545 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.795621 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:47.795746 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.796213 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.797153 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.797421 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.797493 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.797668 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:47.797702 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:47.797749 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.797807 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.797828 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:40:47.797920 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.795201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42079 (local address 127.0.0.1:54890)
0619 02:40:47.795278 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.795290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.795346 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.795631 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.795632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.795634 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:47.795685 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.795688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.796294 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.796295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.797062 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.797065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.797491 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.797492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.797552 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.797566 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:47.797748 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:47.797754 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.797795 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.797847 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:40:47.799036 9125 raft_consensus.cc:1261] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.799126 9265 raft_consensus.cc:1261] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.799275 9353 consensus_queue.cc:1035] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.799367 9345 consensus_queue.cc:1035] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.799847 9340 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: TxnStatusTablet state changed. Reason: New leader 8dde625c6fd445f8853e5e961ea3ab6f. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.799945 9340 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.799903 9341 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: TxnStatusTablet state changed. Reason: New leader 8dde625c6fd445f8853e5e961ea3ab6f. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.800026 9341 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.800709 9342 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.800879 9342 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:47.800932 9342 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.800968 9342 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P 8dde625c6fd445f8853e5e961ea3ab6f: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:47.801120 9341 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.801211 9341 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P cfe55ce640824513accde19f6faaea8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.801259 9340 tablet_replica.cc:439] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dde625c6fd445f8853e5e961ea3ab6f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42079 } } peers { permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 } } peers { permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 } } }
I20260619 02:40:47.801337 9340 tablet_replica.cc:442] T 1c930ea3ce684ee197d35e5a50c2312b P d9c69039fce24e15a76b8b41f9e8cafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:47.807425 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.805096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:39287 (local address 127.7.137.1:42079)
0619 02:40:47.805128 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.805130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.805140 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:40:47.805161 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.805162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.805170 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.805221 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:47.805352 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.805776 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.806845 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.807013 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.807064 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.807232 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:47.807255 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:47.807287 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.807344 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.807363 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260619 02:40:47.807503 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.804896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42079 (local address 127.7.137.1:39287)
0619 02:40:47.805054 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:47.805066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:47.805118 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:47.805228 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:47.805229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:47.805231 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:47.805278 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.805279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.805851 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.805851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.806757 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:47.806768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.807088 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.807089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:47.807144 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.807158 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:47.807302 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:47.807303 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:40:47.807344 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 02:40:47.807385 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260619 02:40:47.809538 9125 raft_consensus.cc:1261] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.809595 9265 raft_consensus.cc:1261] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Refusing update from remote peer 8dde625c6fd445f8853e5e961ea3ab6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:47.809775 9353 consensus_queue.cc:1035] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c69039fce24e15a76b8b41f9e8cafc" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.809880 9345 consensus_queue.cc:1035] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe55ce640824513accde19f6faaea8a" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:47.820032 8827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:47.820140 8827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:47.820760 8827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e0754151dee469dbe1fae88c017a578: Sending DeleteTablet for 3 replicas of tablet a72d37b0d7474749bb26e5f0bf2744b1
I20260619 02:40:47.820998 9245 tablet_service.cc:1514] Processing DeleteTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39638
I20260619 02:40:47.821024 9105 tablet_service.cc:1514] Processing DeleteTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:38402
I20260619 02:40:47.821100 9359 tablet_replica.cc:330] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: stopping tablet replica
I20260619 02:40:47.821120 9360 tablet_replica.cc:330] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: stopping tablet replica
I20260619 02:40:47.821125 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet a72d37b0d7474749bb26e5f0bf2744b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:47 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:54868
I20260619 02:40:47.821163 9359 raft_consensus.cc:2229] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:47.821195 9360 raft_consensus.cc:2229] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:47.821241 9359 raft_consensus.cc:2258] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.821261 9360 raft_consensus.cc:2258] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.821336 9358 tablet_replica.cc:330] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: stopping tablet replica
I20260619 02:40:47.821395 9358 raft_consensus.cc:2229] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:47.821465 9358 raft_consensus.cc:2258] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:47.821712 9360 ts_tablet_manager.cc:1905] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.821815 9358 ts_tablet_manager.cc:1905] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.823421 9360 ts_tablet_manager.cc:1918] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:47.823408 9358 ts_tablet_manager.cc:1918] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:47.823480 9360 log.cc:1196] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/a72d37b0d7474749bb26e5f0bf2744b1
I20260619 02:40:47.823480 9358 log.cc:1196] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/a72d37b0d7474749bb26e5f0bf2744b1
I20260619 02:40:47.823701 9360 ts_tablet_manager.cc:1939] T a72d37b0d7474749bb26e5f0bf2744b1 P d9c69039fce24e15a76b8b41f9e8cafc: Deleting consensus metadata
I20260619 02:40:47.823700 9358 ts_tablet_manager.cc:1939] T a72d37b0d7474749bb26e5f0bf2744b1 P 8dde625c6fd445f8853e5e961ea3ab6f: Deleting consensus metadata
I20260619 02:40:47.824000 8813 catalog_manager.cc:4841] TS d9c69039fce24e15a76b8b41f9e8cafc (127.7.137.2:36661): tablet a72d37b0d7474749bb26e5f0bf2744b1 (table test-table [id=75ddb7f6bbc24436b077871a2afe7f87]) successfully deleted
I20260619 02:40:47.824007 8812 catalog_manager.cc:4841] TS 8dde625c6fd445f8853e5e961ea3ab6f (127.7.137.1:42079): tablet a72d37b0d7474749bb26e5f0bf2744b1 (table test-table [id=75ddb7f6bbc24436b077871a2afe7f87]) successfully deleted
I20260619 02:40:47.824172 9359 ts_tablet_manager.cc:1905] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:47.825704 9359 ts_tablet_manager.cc:1918] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:47.825766 9359 log.cc:1196] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/a72d37b0d7474749bb26e5f0bf2744b1
I20260619 02:40:47.825978 9359 ts_tablet_manager.cc:1939] T a72d37b0d7474749bb26e5f0bf2744b1 P cfe55ce640824513accde19f6faaea8a: Deleting consensus metadata
I20260619 02:40:47.826303 8815 catalog_manager.cc:4841] TS cfe55ce640824513accde19f6faaea8a (127.7.137.3:42477): tablet a72d37b0d7474749bb26e5f0bf2744b1 (table test-table [id=75ddb7f6bbc24436b077871a2afe7f87]) successfully deleted
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[8783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836847, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:40:47.831037 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.821997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.7.137.1:42079)
0619 02:40:47.822041 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.822044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.822213 (+ 169us) server_negotiation.cc:407] Connection header received
0619 02:40:47.822247 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.822248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.822259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.822318 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.823029 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.823673 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.824812 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.825005 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.825055 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.828247 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.828256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.828257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.828264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.829895 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.830252 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.830258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.830262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.830312 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.830522 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.830523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.830523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.830659 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.830814 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.830865 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.830940 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 02:40:47.831286 9024 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54896
I20260619 02:40:47.838640 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:47.832228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.7.137.62:36949)
0619 02:40:47.832268 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:47.832270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:47.832332 (+ 62us) server_negotiation.cc:407] Connection header received
0619 02:40:47.832385 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:47.832386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:47.832396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:47.832455 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:47.832979 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.833776 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.834415 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:47.834726 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:47.834803 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:47.836021 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:47.836022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:47.836022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:47.836030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:47.837635 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.837943 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.837944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.837944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.838002 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:47.838166 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:47.838167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:47.838167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:47.838293 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:47.838392 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:47.838471 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:40:47.838549 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:40:47.838851 8827 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57926
I20260619 02:40:47.839495 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8900
I20260619 02:40:47.845597 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9040
I20260619 02:40:47.849884 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9180
I20260619 02:40:47.854110 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 8796
2026-06-19T02:40:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4460 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[9394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[9394](info): set up 2 sockets
Jun 19 02:40:47 dist-test-slave-pdnb krb5kdc[9394](info): commencing operation
krb5kdc: starting...
W20260619 02:40:48.892335 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:48 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:49.005568 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33879
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:33879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:49.086163 9407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:49.086360 9407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:49.086382 9407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:49.087525 9407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:49.087567 9407 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:40:49.087581 9407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:49.087594 9407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:49.087607 9407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:49.087657 9407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:49.089118 9407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:33879
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:49.089365 9407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:49.089600 9407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:49.094551 9407 server_base.cc:1034] running on GCE node
W20260619 02:40:49.094535 9415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.094535 9412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.094527 9413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.095059 9407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:49.095392 9407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:49.096575 9407 hybrid_clock.cc:648] HybridClock initialized: now 1781836849096555 us; error 44 us; skew 500 ppm
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:49.098980 9407 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:49.099424 9407 webserver.cc:460] Webserver started at http://127.7.137.62:34757/ using document root <none> and password file <none>
I20260619 02:40:49.099658 9407 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:49.099720 9407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:49.099869 9407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:49.100971 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "e0b1537b687d406293920134bcd3f430"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "51ddc8f42011c5f6dffdbb0856712f97"
server_key_iv: "2c978f036ef34bbe664a747979afdc49"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.101348 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "e0b1537b687d406293920134bcd3f430"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "51ddc8f42011c5f6dffdbb0856712f97"
server_key_iv: "2c978f036ef34bbe664a747979afdc49"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.102609 9407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:49.103259 9422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.103425 9407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:49.103519 9407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "e0b1537b687d406293920134bcd3f430"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "51ddc8f42011c5f6dffdbb0856712f97"
server_key_iv: "2c978f036ef34bbe664a747979afdc49"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.103601 9407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:49.116293 9407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:49.118063 9407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:49.118221 9407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:49.122685 9407 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:33879
I20260619 02:40:49.122787 9484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:33879 every 8 connection(s)
I20260619 02:40:49.123075 9407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:49.123699 9485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.125955 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Bootstrap starting.
I20260619 02:40:49.126520 9485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.126802 9485 log.cc:824] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:49.127477 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: No bootstrap required, opened a new log
I20260619 02:40:49.129571 9485 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } }
I20260619 02:40:49.129703 9485 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.129726 9485 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b1537b687d406293920134bcd3f430, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.129892 9485 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } }
I20260619 02:40:49.129954 9485 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:49.129995 9485 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:49.130045 9485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:49.130192 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9407
I20260619 02:40:49.130311 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:49.130468 7716 external_mini_cluster.cc:1428] Setting key 7bf7e2de0a3befdcf5d791227c5b05bd
I20260619 02:40:49.130666 9485 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } }
I20260619 02:40:49.130777 9485 leader_election.cc:302] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0b1537b687d406293920134bcd3f430; no voters:
I20260619 02:40:49.131122 9485 leader_election.cc:288] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:49.131171 9490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:49.131315 9490 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [term 1 LEADER]: Becoming Leader. State: Replica: e0b1537b687d406293920134bcd3f430, State: Running, Role: LEADER
I20260619 02:40:49.131379 9485 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:49.131451 9490 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } }
I20260619 02:40:49.131857 9490 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b1537b687d406293920134bcd3f430. Latest consensus state: current_term: 1 leader_uuid: "e0b1537b687d406293920134bcd3f430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } } }
I20260619 02:40:49.132001 9490 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:49.132266 9499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:49.133008 9499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:49.134122 9492 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0b1537b687d406293920134bcd3f430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1537b687d406293920134bcd3f430" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33879 } } }
I20260619 02:40:49.135311 9499 catalog_manager.cc:1347] Generated new cluster ID: 82fd98b54d7e4220a7db4bd3102d3a78
I20260619 02:40:49.135609 9492 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:49.135637 9499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.141865 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.130925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.7.137.62:33879)
0619 02:40:49.131080 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.131091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.131127 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:40:49.131340 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.131347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.131403 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.131548 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.132325 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.133733 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.134638 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.135270 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.135373 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.138603 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.138607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.138607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.138619 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.140272 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.140643 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.140647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.140647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.140801 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.141011 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.141012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.141012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.141271 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.141429 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.141523 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.141655 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260619 02:40:49.162428 9499 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:49.162953 9499 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:49.167011 9499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Generated new TSK 0
I20260619 02:40:49.167191 9499 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:49.195956 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33879
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:49.296572 9511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:49.296793 9511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:49.296825 9511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:49.297885 9511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:49.297919 9511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:49.297938 9511 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:40:49.297950 9511 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:49.297966 9511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:49.299291 9511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:49.299458 9511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:49.299619 9511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:49.304482 9517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.304520 9516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.304520 9519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.304778 9511 server_base.cc:1034] running on GCE node
I20260619 02:40:49.304950 9511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:49.305140 9511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:49.306270 9511 hybrid_clock.cc:648] HybridClock initialized: now 1781836849306258 us; error 27 us; skew 500 ppm
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:49.308825 9511 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:49.309195 9511 webserver.cc:460] Webserver started at http://127.7.137.1:38851/ using document root <none> and password file <none>
I20260619 02:40:49.309423 9511 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:49.309492 9511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:49.309640 9511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:49.310739 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "bc83d5a3e436d36011b997f1575659da"
server_key_iv: "1b2b9c79236e9a4aa10733501d25cf46"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.311094 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "bc83d5a3e436d36011b997f1575659da"
server_key_iv: "1b2b9c79236e9a4aa10733501d25cf46"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.312418 9511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:40:49.313259 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.313457 9511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:49.313558 9511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "bc83d5a3e436d36011b997f1575659da"
server_key_iv: "1b2b9c79236e9a4aa10733501d25cf46"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.313630 9511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:49.322047 9511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:49.324254 9511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:49.324416 9511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:49.324949 9511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:49.325007 9511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.325052 9511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:49.325075 9511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.331784 9511 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:42105
I20260619 02:40:49.331837 9641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:42105 every 8 connection(s)
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.332293 9511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:40:49.335120 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.325394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:54185 (local address 127.7.137.62:33879)
0619 02:40:49.325439 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.325441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.325638 (+ 197us) server_negotiation.cc:407] Connection header received
0619 02:40:49.326180 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.326181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.326195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.326247 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.327069 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.327929 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.329529 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.329889 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.329975 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.332575 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.332577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.332577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.332588 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.333964 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.334354 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.334355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.334355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.334411 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.334636 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.334636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.334637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.334744 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.334794 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.334993 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.335058 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:49.335395 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.325375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.1:54185)
0619 02:40:49.325517 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.325551 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.326056 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.326350 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.326354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.326668 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.326958 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.326966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.328025 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.328026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.329419 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.329422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.329971 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.329972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.330104 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.330625 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.330635 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.332452 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.334033 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.334039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.334041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.334262 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.334479 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.334480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.334480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.334577 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.334797 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.334801 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.334906 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.334982 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.335098 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260619 02:40:49.335914 9642 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33879
I20260619 02:40:49.335997 9642 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:49.336179 9642 heartbeater.cc:511] Master 127.7.137.62:33879 requested a full tablet report, sending...
I20260619 02:40:49.336650 9438 ts_manager.cc:194] Registered new tserver with Master: 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105)
I20260619 02:40:49.337524 9438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:54185
I20260619 02:40:49.341203 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9511
I20260619 02:40:49.341349 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:49.341572 7716 external_mini_cluster.cc:1428] Setting key 96a9ff89ce1cf94a3b93bddb7d7c73f0
I20260619 02:40:49.342499 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.336993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:43125 (local address 127.7.137.62:33879)
0619 02:40:49.337036 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.337038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.337112 (+ 74us) server_negotiation.cc:407] Connection header received
0619 02:40:49.337167 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.337168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.337177 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.337236 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.337491 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.338091 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.338799 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.338971 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.339022 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.339914 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.339915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.339915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.339921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.341325 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.341763 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.341765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.341765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.341822 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.342039 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.342040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.342040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.342151 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.342228 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.342373 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.342442 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:40:49.342641 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.336933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.1:43125)
0619 02:40:49.337052 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.337062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.337116 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.337238 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.337238 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.337379 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.337428 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.337429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.338186 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.338187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.338728 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.338730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.339032 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.339033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.339084 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.339520 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.339525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.339809 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.341427 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.341429 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.341429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.341650 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.341900 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.341900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.341901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.341966 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.342223 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.342224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.342270 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.342340 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.342446 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:49.365887 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33879
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:49.444711 9651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:49.444986 9651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:49.445025 9651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:49.446138 9651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:49.446202 9651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:49.446236 9651 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:40:49.446267 9651 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:49.446301 9651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:49.447714 9651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:49.447974 9651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:49.448201 9651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:49.453006 9659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.453050 9651 server_base.cc:1034] running on GCE node
W20260619 02:40:49.453012 9656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.453004 9657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.453429 9651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:49.453663 9651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:49.454828 9651 hybrid_clock.cc:648] HybridClock initialized: now 1781836849454822 us; error 34 us; skew 500 ppm
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:49.457190 9651 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:49.457625 9651 webserver.cc:460] Webserver started at http://127.7.137.2:39911/ using document root <none> and password file <none>
I20260619 02:40:49.457849 9651 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:49.457916 9651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:49.458045 9651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:49.459026 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "380e5dc39b7b466eac52740fcea0debb"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "5202f07a206e8f23696088c5695db0e3"
server_key_iv: "1e74ef6386adef4dcf03cc00b5c7fa75"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.459378 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "380e5dc39b7b466eac52740fcea0debb"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "5202f07a206e8f23696088c5695db0e3"
server_key_iv: "1e74ef6386adef4dcf03cc00b5c7fa75"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.460704 9651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:40:49.461443 9666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.461598 9651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:49.461699 9651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "380e5dc39b7b466eac52740fcea0debb"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "5202f07a206e8f23696088c5695db0e3"
server_key_iv: "1e74ef6386adef4dcf03cc00b5c7fa75"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.461783 9651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:49.469761 9651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:49.471359 9651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:49.471505 9651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:49.471943 9651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:49.472000 9651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.472069 9651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:49.472100 9651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.479048 9651 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:33627
I20260619 02:40:49.479118 9781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:33627 every 8 connection(s)
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.479432 9651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:49.480000 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9651
I20260619 02:40:49.480096 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:49.480283 7716 external_mini_cluster.cc:1428] Setting key 7828da500a44a509434aa2ef43779ac9
I20260619 02:40:49.482820 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.472474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:35051 (local address 127.7.137.62:33879)
0619 02:40:49.472541 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.472543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.472786 (+ 243us) server_negotiation.cc:407] Connection header received
0619 02:40:49.473461 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.473462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.473475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.473544 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.474395 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.475214 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.476392 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.476699 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.476783 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.479995 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.479996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.479997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.480004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.481476 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.481891 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.481892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.481893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.481954 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.482196 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.482196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.482197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.482315 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.482391 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.482648 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.482723 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:40:49.483011 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.472445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.2:35051)
0619 02:40:49.472634 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.472681 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.473351 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.473611 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.473615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.474010 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.474301 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.474307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.475302 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.475303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.476278 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.476280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.476850 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.476851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.477019 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.478093 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.478107 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.479893 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.481578 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.481584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.481587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.481802 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.482018 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.482019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.482019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.482129 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.482381 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.482386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.482521 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.482627 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.482780 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260619 02:40:49.483492 9782 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33879
I20260619 02:40:49.483573 9782 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:49.483759 9782 heartbeater.cc:511] Master 127.7.137.62:33879 requested a full tablet report, sending...
I20260619 02:40:49.485231 9438 ts_manager.cc:194] Registered new tserver with Master: 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627)
I20260619 02:40:49.485764 9438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:35051
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:40:49.490628 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.484585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:43363 (local address 127.7.137.62:33879)
0619 02:40:49.485148 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.485150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.485156 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:49.485173 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.485174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.485182 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.485257 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.485605 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.486224 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.486965 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.487163 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.487244 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.488088 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.488089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.488090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.488096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.489483 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.489798 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.489799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.489799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.489849 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.490050 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.490050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.490051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.490191 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.490268 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.490474 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.490540 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539}
I20260619 02:40:49.490634 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.484524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.2:43363)
0619 02:40:49.484695 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.484729 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.484781 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.485310 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.485311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.485472 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.485516 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.485517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.486312 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.486313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.486866 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.486868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.487251 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.487251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.487327 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.487747 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.487751 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.488001 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.489547 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.489548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.489548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.489715 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.489922 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.489922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.489923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.489979 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.490290 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.490290 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.490334 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.490438 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.490541 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:49.505240 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33879
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:49.584594 9791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:49.584864 9791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:49.584904 9791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:49.586014 9791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:49.586081 9791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:49.586114 9791 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:40:49.586143 9791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:40:49.586179 9791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:49.587576 9791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:49.587831 9791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:49.588058 9791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:49.592993 9797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:49.592985 9799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.593014 9791 server_base.cc:1034] running on GCE node
W20260619 02:40:49.592989 9796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:49.593398 9791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:49.593607 9791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:49.594738 9791 hybrid_clock.cc:648] HybridClock initialized: now 1781836849594729 us; error 30 us; skew 500 ppm
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:49.597345 9791 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:49.597785 9791 webserver.cc:460] Webserver started at http://127.7.137.3:40985/ using document root <none> and password file <none>
I20260619 02:40:49.597970 9791 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:49.598008 9791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:49.598096 9791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:49.599073 9791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "be95382fe5594300a624dd27657faa98"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "c6b689173e890f768daa73d26369bc08"
server_key_iv: "ba4d8edda77168bcf7dcaa3b6f800096"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.599367 9791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "be95382fe5594300a624dd27657faa98"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "c6b689173e890f768daa73d26369bc08"
server_key_iv: "ba4d8edda77168bcf7dcaa3b6f800096"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.600557 9791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:49.601303 9806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.601464 9791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.601567 9791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "be95382fe5594300a624dd27657faa98"
format_stamp: "Formatted at 2026-06-19 02:40:49 on dist-test-slave-pdnb"
server_key: "c6b689173e890f768daa73d26369bc08"
server_key_iv: "ba4d8edda77168bcf7dcaa3b6f800096"
server_key_version: "encryptionkey@0"
I20260619 02:40:49.601636 9791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:49.629873 9791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:49.632339 9791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:49.632496 9791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:49.633162 9791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:49.633230 9791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.633260 9791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:49.633275 9791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.640632 9791 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:42461
I20260619 02:40:49.640823 9921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:42461 every 8 connection(s)
I20260619 02:40:49.641109 9791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:49.643512 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.633580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:55533 (local address 127.7.137.62:33879)
0619 02:40:49.633627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.633629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.633983 (+ 354us) server_negotiation.cc:407] Connection header received
0619 02:40:49.634728 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.634729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.634743 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.634826 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.635787 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.636397 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.637591 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.637810 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.637897 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.640358 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.640360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.640360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.640370 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.642171 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.642576 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.642578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.642579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.642640 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.642878 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.642879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.642879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.642998 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.643067 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.643341 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.643418 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:49.643708 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.633603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.3:55533)
0619 02:40:49.633790 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.633854 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.634570 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.634921 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.634926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.635354 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.635677 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.635685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.636498 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.636499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.637487 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.637490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.637919 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.637920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.638037 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.638564 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.638575 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.640256 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.642256 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.642262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.642265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.642484 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.642705 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.642706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.642706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.642811 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.643068 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.643075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.643207 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.643320 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.643485 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260619 02:40:49.644176 9922 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33879
I20260619 02:40:49.644258 9922 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:49.644431 9922 heartbeater.cc:511] Master 127.7.137.62:33879 requested a full tablet report, sending...
I20260619 02:40:49.644835 9438 ts_manager.cc:194] Registered new tserver with Master: be95382fe5594300a624dd27657faa98 (127.7.137.3:42461)
I20260619 02:40:49.645198 9438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:55533
I20260619 02:40:49.649828 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 9791
I20260619 02:40:49.649897 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:49.650087 7716 external_mini_cluster.cc:1428] Setting key ec9ca33d14a3255ca78059f849439622
I20260619 02:40:49.650724 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.645516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:59715 (local address 127.7.137.62:33879)
0619 02:40:49.645555 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.645556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.645565 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:49.645642 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.645642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.645655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.645693 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.646064 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.646515 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.647159 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.647336 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.647388 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.648155 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.648156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.648156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.648161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.649396 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.649828 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.649829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.649830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.649910 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.650116 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.650117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.650118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.650267 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.650337 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.650547 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.650626 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:49.650732 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.645374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.7.137.3:59715)
0619 02:40:49.645518 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.645531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.645576 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.645700 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.645700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.645895 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.645971 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.645973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.646601 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.646601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.647091 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.647092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.647397 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.647397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.647448 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.647825 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.647829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.648084 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.649474 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.649475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.649475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.649640 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.649967 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.649968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.649968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.650028 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.650379 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.650380 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.650424 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.650517 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.650637 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260619 02:40:49.651072 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.672828 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.666082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.7.137.62:33879)
0619 02:40:49.666134 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.666137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.666145 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:49.666217 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.666218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.666234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.666296 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.666855 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.667351 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.668087 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.668267 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.668353 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.670548 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.670549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.670550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.670556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.671712 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.672065 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.672066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.672067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.672123 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.672315 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.672316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.672316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.672422 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.672526 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.672623 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.672700 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:40:49.674219 9438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:49.675736 9438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:49.682026 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.678015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42105 (local address 127.0.0.1:39872)
0619 02:40:49.678326 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.678342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.678408 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.679069 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.679070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.679072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:49.679113 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.679114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.679939 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.679940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.680977 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.680979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.681804 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.681805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.681860 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.681871 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.681929 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.681964 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":72,"threads_started":1}
I20260619 02:40:49.682096 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.678114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.7.137.1:42105)
0619 02:40:49.678925 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.678929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.678937 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:49.678976 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.678978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.678989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.679072 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:49.679169 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.679856 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.681059 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.681722 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.681798 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.681853 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.681961 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.682000 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":91,"threads_started":1}
I20260619 02:40:49.682809 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.677993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.0.0.1:47238)
0619 02:40:49.678296 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.678339 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.678408 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.678654 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.678657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.678658 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:49.678719 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.678720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.679962 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.679963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.681279 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.681282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.682524 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.682526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.682604 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.682608 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.682697 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.682738 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":71,"thread_start_us":68,"threads_started":1}
I20260619 02:40:49.682875 9576 tablet_service.cc:1467] Processing CreateTablet for tablet cf6246a2bb834452b0228589f572b36b (DEFAULT_TABLE table=test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:49.683171 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.678128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.7.137.3:42461)
0619 02:40:49.678367 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.678372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.678384 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:40:49.678522 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.678524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.678539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.678653 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:49.678778 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.679864 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.681367 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.682103 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.682174 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.682229 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.682712 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.682750 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260619 02:40:49.683255 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.678216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33627 (local address 127.0.0.1:55188)
0619 02:40:49.678427 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.678435 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.678474 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.678742 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.678743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.678745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:49.678791 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.678793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.680011 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.680012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.681287 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.681288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.683031 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.683032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.683096 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.683100 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.683149 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.683191 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":87,"thread_start_us":67,"threads_started":1}
I20260619 02:40:49.683424 9576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6246a2bb834452b0228589f572b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.683530 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.678335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.7.137.2:33627)
0619 02:40:49.678516 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.678520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.678529 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:49.678569 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.678571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.678600 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.678675 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:49.678856 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.679915 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.682264 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.682880 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.682943 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.682999 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.683377 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.683425 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260619 02:40:49.683838 9839 tablet_service.cc:1467] Processing CreateTablet for tablet cf6246a2bb834452b0228589f572b36b (DEFAULT_TABLE table=test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:49.684134 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6246a2bb834452b0228589f572b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.684065 9716 tablet_service.cc:1467] Processing CreateTablet for tablet cf6246a2bb834452b0228589f572b36b (DEFAULT_TABLE table=test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:49.684302 9716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6246a2bb834452b0228589f572b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.687207 9945 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Bootstrap starting.
I20260619 02:40:49.687393 9946 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Bootstrap starting.
I20260619 02:40:49.687829 9945 tablet_bootstrap.cc:654] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.687951 9946 tablet_bootstrap.cc:654] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.688153 9945 log.cc:824] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:49.688277 9946 log.cc:824] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:49.688593 9947 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Bootstrap starting.
I20260619 02:40:49.688946 9945 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: No bootstrap required, opened a new log
I20260619 02:40:49.689044 9945 ts_tablet_manager.cc:1397] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:40:49.689467 9947 tablet_bootstrap.cc:654] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.689767 9947 log.cc:824] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:49.690291 9946 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: No bootstrap required, opened a new log
I20260619 02:40:49.690366 9946 ts_tablet_manager.cc:1397] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:40:49.690495 9947 tablet_bootstrap.cc:492] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: No bootstrap required, opened a new log
I20260619 02:40:49.690567 9947 ts_tablet_manager.cc:1397] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:40:49.690583 9945 raft_consensus.cc:348] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.690743 9945 raft_consensus.cc:374] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.690815 9945 raft_consensus.cc:729] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.690951 9945 consensus_queue.cc:260] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.691213 9945 ts_tablet_manager.cc:1428] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:40:49.691356 9782 heartbeater.cc:503] Master 127.7.137.62:33879 was elected leader, sending a full tablet report...
I20260619 02:40:49.692344 9946 raft_consensus.cc:348] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.692533 9946 raft_consensus.cc:374] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.692616 9946 raft_consensus.cc:729] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.692605 9947 raft_consensus.cc:348] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.692817 9947 raft_consensus.cc:374] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.692765 9946 consensus_queue.cc:260] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.692866 9947 raft_consensus.cc:729] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be95382fe5594300a624dd27657faa98, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.692977 9947 consensus_queue.cc:260] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.693045 9946 ts_tablet_manager.cc:1428] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:40:49.693106 9642 heartbeater.cc:503] Master 127.7.137.62:33879 was elected leader, sending a full tablet report...
I20260619 02:40:49.693245 9947 ts_tablet_manager.cc:1428] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:40:49.693288 9922 heartbeater.cc:503] Master 127.7.137.62:33879 was elected leader, sending a full tablet report...
W20260619 02:40:49.730085 9783 tablet.cc:2369] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:49.826133 9951 raft_consensus.cc:482] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:49.826272 9951 raft_consensus.cc:504] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.826685 9951 leader_election.cc:288] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be95382fe5594300a624dd27657faa98 (127.7.137.3:42461), 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105)
I20260619 02:40:49.830621 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.826960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:59145 (local address 127.7.137.1:42105)
0619 02:40:49.827020 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.827022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.827033 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:40:49.827131 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.827132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.827150 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.827200 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:49.827380 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.828159 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.829707 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.830258 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.830326 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.830335 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.830513 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.830551 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:40:49.830621 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.826959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:40823 (local address 127.7.137.3:42461)
0619 02:40:49.827005 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.827007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.827286 (+ 279us) server_negotiation.cc:407] Connection header received
0619 02:40:49.827325 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.827326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.827356 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.827413 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:49.827529 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.828214 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.829707 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.830248 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.830326 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.830335 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.830513 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.830553 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:40:49.830636 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.826900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42105 (local address 127.7.137.2:59145)
0619 02:40:49.826970 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.826983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.827037 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.827223 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.827223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.827225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:49.827300 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.827302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.828262 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.828263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.829620 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.829623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.830345 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.830346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.830427 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.830434 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.830496 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.830545 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":97}
I20260619 02:40:49.830646 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.826906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.7.137.2:40823)
0619 02:40:49.827144 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.827160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.827273 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.827424 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.827424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.827425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:49.827468 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.827469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.828279 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.828279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.829620 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.829623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.830345 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.830346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.830427 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.830434 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.830496 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.830545 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":121,"thread_start_us":114,"threads_started":1}
I20260619 02:40:49.831072 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6246a2bb834452b0228589f572b36b" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" is_pre_election: true
I20260619 02:40:49.831079 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6246a2bb834452b0228589f572b36b" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98" is_pre_election: true
I20260619 02:40:49.831214 9596 raft_consensus.cc:2454] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 0.
I20260619 02:40:49.831279 9876 raft_consensus.cc:2454] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 0.
I20260619 02:40:49.831487 9668 leader_election.cc:302] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380e5dc39b7b466eac52740fcea0debb, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:49.831606 9951 raft_consensus.cc:2781] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:49.831660 9951 raft_consensus.cc:482] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:49.831693 9951 raft_consensus.cc:3037] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:49.832748 9951 raft_consensus.cc:504] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.832887 9951 leader_election.cc:288] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 election: Requested vote from peers be95382fe5594300a624dd27657faa98 (127.7.137.3:42461), 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105)
W20260619 02:40:49.833032 9643 tablet.cc:2369] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:49.833082 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6246a2bb834452b0228589f572b36b" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
I20260619 02:40:49.833173 9596 raft_consensus.cc:3037] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:49.833467 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6246a2bb834452b0228589f572b36b" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98"
I20260619 02:40:49.833546 9876 raft_consensus.cc:3037] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:49.834040 9596 raft_consensus.cc:2454] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 1.
I20260619 02:40:49.834300 9667 leader_election.cc:302] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, 380e5dc39b7b466eac52740fcea0debb; no voters:
I20260619 02:40:49.834374 9876 raft_consensus.cc:2454] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 1.
I20260619 02:40:49.834461 9951 raft_consensus.cc:2781] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:49.834650 9951 raft_consensus.cc:686] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 1 LEADER]: Becoming Leader. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Running, Role: LEADER
I20260619 02:40:49.834756 9951 consensus_queue.cc:237] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } }
I20260619 02:40:49.835528 9438 catalog_manager.cc:5495] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb reported cstate change: term changed from 0 to 1, leader changed from <none> to 380e5dc39b7b466eac52740fcea0debb (127.7.137.2). New cstate: current_term: 1 leader_uuid: "380e5dc39b7b466eac52740fcea0debb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } health_report { overall_health: HEALTHY } } }
Jun 19 02:40:49 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836849, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:49.884145 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.877449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.7.137.62:33879)
0619 02:40:49.877568 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:49.877570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:49.877674 (+ 104us) server_negotiation.cc:407] Connection header received
0619 02:40:49.877763 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:49.877764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:49.877774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:49.877831 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:49.878308 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.878798 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.879485 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.879669 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.879757 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.881808 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:49.881809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:49.881810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:49.881816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:49.882967 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.883357 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.883358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.883359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.883415 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:49.883600 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:49.883601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:49.883601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:49.883710 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:49.883784 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:49.883934 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:40:49.884039 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:40:49.884214 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:49.877372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33879 (local address 127.0.0.1:36460)
0619 02:40:49.877592 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:49.877603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:49.877695 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:49.877841 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:49.877842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:49.878153 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:49.878194 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.878197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.878875 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.878876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.879393 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:49.879395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:49.879759 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:49.879759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:49.879811 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:49.880350 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:49.880355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:49.881687 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:49.883059 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.883066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.883069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.883281 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.883479 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:49.883480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:49.883480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:49.883539 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:49.883776 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:49.883782 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:49.883826 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:40:49.883912 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:40:49.884095 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":90,"thread_start_us":114,"threads_started":1}
I20260619 02:40:49.886025 9438 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:36460:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:40:49.886225 9438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:49.887660 9839 tablet_service.cc:1467] Processing CreateTablet for tablet 0af06ffea1f6434583b2fa4bda028675 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=291dabb50ab443f286f39d4a138f5592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:49.887646 9576 tablet_service.cc:1467] Processing CreateTablet for tablet 0af06ffea1f6434583b2fa4bda028675 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=291dabb50ab443f286f39d4a138f5592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:49.887702 9716 tablet_service.cc:1467] Processing CreateTablet for tablet 0af06ffea1f6434583b2fa4bda028675 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=291dabb50ab443f286f39d4a138f5592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:40:49.887814 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0af06ffea1f6434583b2fa4bda028675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.887815 9576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0af06ffea1f6434583b2fa4bda028675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.887888 9716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0af06ffea1f6434583b2fa4bda028675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:49.889008 9945 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: Bootstrap starting.
I20260619 02:40:49.889285 9947 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: Bootstrap starting.
I20260619 02:40:49.889446 9945 tablet_bootstrap.cc:654] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.889842 9947 tablet_bootstrap.cc:654] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.890178 9946 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Bootstrap starting.
I20260619 02:40:49.890244 9945 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: No bootstrap required, opened a new log
I20260619 02:40:49.890290 9945 ts_tablet_manager.cc:1397] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:49.890460 9945 raft_consensus.cc:348] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.890547 9945 raft_consensus.cc:374] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.890560 9946 tablet_bootstrap.cc:654] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:49.890570 9945 raft_consensus.cc:729] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.890626 9945 consensus_queue.cc:260] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.890740 9945 ts_tablet_manager.cc:1428] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:49.890861 9951 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:49.890938 9951 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:49.891189 9946 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: No bootstrap required, opened a new log
I20260619 02:40:49.891239 9946 ts_tablet_manager.cc:1397] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:49.891492 9947 tablet_bootstrap.cc:492] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: No bootstrap required, opened a new log
I20260619 02:40:49.891456 9946 raft_consensus.cc:348] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.891523 9946 raft_consensus.cc:374] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.891539 9946 raft_consensus.cc:729] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.891539 9947 ts_tablet_manager.cc:1397] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 02:40:49.891595 9946 consensus_queue.cc:260] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.891695 9946 ts_tablet_manager.cc:1428] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 02:40:49.891702 9923 tablet.cc:2369] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:40:49.891700 9947 raft_consensus.cc:348] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.891728 9952 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:49.891808 9947 raft_consensus.cc:374] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:49.891813 9952 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:49.891834 9947 raft_consensus.cc:729] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be95382fe5594300a624dd27657faa98, State: Initialized, Role: FOLLOWER
I20260619 02:40:49.891875 9947 consensus_queue.cc:260] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:49.891960 9947 ts_tablet_manager.cc:1428] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:49.892009 9953 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:49.892076 9953 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:50.276140 9953 raft_consensus.cc:482] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:50.276242 9953 raft_consensus.cc:504] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.276557 9953 leader_election.cc:288] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627)
I20260619 02:40:50.279781 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.276970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:44265 (local address 127.7.137.1:42105)
0619 02:40:50.277045 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.277047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.277060 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:40:50.277115 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.277116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.277126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.277208 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:50.277421 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.277956 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279084 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.279423 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279508 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.279518 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.279668 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.279701 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:40:50.279788 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.277158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:48195 (local address 127.7.137.2:33627)
0619 02:40:50.277286 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.277289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.277304 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:40:50.277329 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.277329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.277340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.277412 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:50.277565 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.278093 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279041 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.279424 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279508 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.279519 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.279679 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.279715 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260619 02:40:50.279799 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.277050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33627 (local address 127.7.137.3:48195)
0619 02:40:50.277164 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.277174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.277224 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.277429 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.277429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.277432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:50.277487 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.277489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.278173 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.278174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.278949 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.278951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.279509 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.279586 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.279589 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.279651 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.279697 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260619 02:40:50.279800 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.276848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42105 (local address 127.7.137.3:44265)
0619 02:40:50.276982 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.276995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.277053 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.277227 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.277228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.277232 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:50.277315 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.277317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.278039 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.278040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.278965 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.278967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.279511 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.279511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.279588 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.279592 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.279661 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.279710 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":39,"thread_start_us":84,"threads_started":1}
I20260619 02:40:50.280107 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0af06ffea1f6434583b2fa4bda028675" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb" is_pre_election: true
I20260619 02:40:50.280237 9736 raft_consensus.cc:2454] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 0.
I20260619 02:40:50.280251 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0af06ffea1f6434583b2fa4bda028675" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" is_pre_election: true
I20260619 02:40:50.280328 9596 raft_consensus.cc:2454] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 0.
I20260619 02:40:50.280445 9807 leader_election.cc:302] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380e5dc39b7b466eac52740fcea0debb, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:50.280607 9953 raft_consensus.cc:2781] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:50.280648 9953 raft_consensus.cc:482] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:50.280681 9953 raft_consensus.cc:3037] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.281387 9953 raft_consensus.cc:504] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.281514 9953 leader_election.cc:288] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 election: Requested vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627)
I20260619 02:40:50.281646 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0af06ffea1f6434583b2fa4bda028675" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
I20260619 02:40:50.281728 9596 raft_consensus.cc:3037] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.281692 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0af06ffea1f6434583b2fa4bda028675" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb"
I20260619 02:40:50.281780 9736 raft_consensus.cc:3037] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.282331 9596 raft_consensus.cc:2454] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 1.
I20260619 02:40:50.282378 9736 raft_consensus.cc:2454] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 1.
I20260619 02:40:50.282541 9807 leader_election.cc:302] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380e5dc39b7b466eac52740fcea0debb, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:50.282639 9953 raft_consensus.cc:2781] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:50.282742 9953 raft_consensus.cc:686] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [term 1 LEADER]: Becoming Leader. State: Replica: be95382fe5594300a624dd27657faa98, State: Running, Role: LEADER
I20260619 02:40:50.282835 9953 consensus_queue.cc:237] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.283283 9969 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: TxnStatusTablet state changed. Reason: New leader be95382fe5594300a624dd27657faa98. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.283386 9969 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:50.283557 9971 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:40:50.283664 9971 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:40:50.283720 9439 catalog_manager.cc:5495] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 reported cstate change: term changed from 0 to 1, leader changed from <none> to be95382fe5594300a624dd27657faa98 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } health_report { overall_health: HEALTHY } } }
I20260619 02:40:50.334203 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.331293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.0.0.1:47248)
0619 02:40:50.331370 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.331383 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.331437 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.331699 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.331700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.331702 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:50.331756 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.331758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.332515 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.332516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.333289 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.333291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.333694 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.333696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.333763 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.333777 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.334037 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.334046 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.334091 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.334137 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:40:50.334203 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.331370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.7.137.3:42461)
0619 02:40:50.331482 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.331484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.331493 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:50.331528 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.331528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.331547 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.331830 (+ 283us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.331862 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.332421 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.333381 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.333590 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.333845 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.333875 (+ 30us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.333960 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.334041 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.334094 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.334114 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":45,"threads_started":1}
I20260619 02:40:50.335994 9596 raft_consensus.cc:1261] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Refusing update from remote peer be95382fe5594300a624dd27657faa98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.336040 9736 raft_consensus.cc:1261] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Refusing update from remote peer be95382fe5594300a624dd27657faa98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.336318 9969 consensus_queue.cc:1035] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:50.336506 9953 consensus_queue.cc:1035] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:50.337558 9951 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: TxnStatusTablet state changed. Reason: New leader be95382fe5594300a624dd27657faa98. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.337647 9951 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:50.337575 9952 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: TxnStatusTablet state changed. Reason: New leader be95382fe5594300a624dd27657faa98. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.337682 9952 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:50.338462 9953 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.338541 9953 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:50.338598 9975 mvcc.cc:205] Tried to move back new op lower bound from 7298403738973777920 to 7298403738759061504. Current Snapshot: MvccSnapshot[applied={T|T < 7298403738973777920}]
I20260619 02:40:50.338915 9969 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.338986 9969 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P be95382fe5594300a624dd27657faa98: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:40:50.339226 9951 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.339308 9951 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 380e5dc39b7b466eac52740fcea0debb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:50.339908 9952 tablet_replica.cc:439] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } } }
I20260619 02:40:50.339990 9952 tablet_replica.cc:442] T 0af06ffea1f6434583b2fa4bda028675 P 2a6f011746cc4bfa9bf7e65efeaa9e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:40:50.340219 9976 mvcc.cc:205] Tried to move back new op lower bound from 7298403738973777920 to 7298403738759061504. Current Snapshot: MvccSnapshot[applied={T|T < 7298403738973777920}]
I20260619 02:40:50.340246 9977 mvcc.cc:205] Tried to move back new op lower bound from 7298403738973777920 to 7298403738759061504. Current Snapshot: MvccSnapshot[applied={T|T < 7298403738973777920}]
I20260619 02:40:50.343529 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.341048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.7.137.2:33627)
0619 02:40:50.341089 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.341091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.341209 (+ 118us) server_negotiation.cc:407] Connection header received
0619 02:40:50.341292 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.341293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.341310 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.341377 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.341486 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.341946 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.342733 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.342942 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.343018 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.343184 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.343281 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.343363 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.343414 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.343435 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:50.351392 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.345814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:47909 (local address 127.7.137.3:42461)
0619 02:40:50.345854 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.345856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.348921 (+ 3065us) server_negotiation.cc:407] Connection header received
0619 02:40:50.348979 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.348980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.348990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.349048 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.349182 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.349650 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.350722 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.350899 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.350954 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.351167 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.351194 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.351235 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.351322 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.351340 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:50.351444 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.345756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.7.137.2:47909)
0619 02:40:50.348841 (+ 3085us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.348854 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.348920 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.349056 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.349057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.349058 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:50.349120 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.349121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.349745 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.349746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.350633 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.350636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.350980 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.350981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.351047 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.351061 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.351259 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.351261 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.351320 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.351365 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3042,"thread_start_us":87,"threads_started":1}
I20260619 02:40:50.353943 9596 raft_consensus.cc:1261] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Refusing update from remote peer 380e5dc39b7b466eac52740fcea0debb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.354045 9876 raft_consensus.cc:1261] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Refusing update from remote peer 380e5dc39b7b466eac52740fcea0debb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.354337 9989 consensus_queue.cc:1035] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Connected to new peer: Peer: permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:50.354483 9951 consensus_queue.cc:1035] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:50.362540 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.359894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:34055 (local address 127.7.137.2:33627)
0619 02:40:50.359933 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.359935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.359943 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:40:50.359964 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.359965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.359975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.360056 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.360203 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.360937 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.361736 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.362007 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.362079 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.362296 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.362329 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.362358 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.362469 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.362489 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:50.362586 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.359638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33627 (local address 127.7.137.3:34055)
0619 02:40:50.359803 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.359813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.359882 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.360064 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.360065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.360067 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:50.360128 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.360131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.361011 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.361011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.361633 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.361635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.362117 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.362118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.362184 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.362198 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.362399 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.362401 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.362448 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.362493 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260619 02:40:50.372159 9439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:50.372282 9439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:50.373051 9439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Sending DeleteTablet for 3 replicas of tablet cf6246a2bb834452b0228589f572b36b
I20260619 02:40:50.373369 9576 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6246a2bb834452b0228589f572b36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:50 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39872
I20260619 02:40:50.373394 9716 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6246a2bb834452b0228589f572b36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:50 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:55188
I20260619 02:40:50.373679 9439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:50.373845 9439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:50.374131 9839 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6246a2bb834452b0228589f572b36b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:50 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47238
I20260619 02:40:50.375073 9716 tablet_service.cc:1467] Processing CreateTablet for tablet ae36ea32719648ee81d5d1b12db7c749 (DEFAULT_TABLE table=test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:50.375094 9576 tablet_service.cc:1467] Processing CreateTablet for tablet ae36ea32719648ee81d5d1b12db7c749 (DEFAULT_TABLE table=test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:50.375229 9576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36ea32719648ee81d5d1b12db7c749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:50.375229 9716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36ea32719648ee81d5d1b12db7c749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:50.376294 9945 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Bootstrap starting.
I20260619 02:40:50.376598 9946 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Bootstrap starting.
I20260619 02:40:50.376590 9839 tablet_service.cc:1467] Processing CreateTablet for tablet ae36ea32719648ee81d5d1b12db7c749 (DEFAULT_TABLE table=test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:50.376698 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae36ea32719648ee81d5d1b12db7c749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:50.376739 9945 tablet_bootstrap.cc:654] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:50.377002 9946 tablet_bootstrap.cc:654] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:50.377655 9946 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: No bootstrap required, opened a new log
I20260619 02:40:50.377708 9946 ts_tablet_manager.cc:1397] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:50.377849 9946 raft_consensus.cc:348] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.377923 9946 raft_consensus.cc:374] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:50.377945 9946 raft_consensus.cc:729] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Initialized, Role: FOLLOWER
I20260619 02:40:50.377986 9946 consensus_queue.cc:260] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.378083 9946 ts_tablet_manager.cc:1428] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:50.378192 9947 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Bootstrap starting.
I20260619 02:40:50.378659 9947 tablet_bootstrap.cc:654] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:50.379308 9947 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: No bootstrap required, opened a new log
I20260619 02:40:50.379369 9947 ts_tablet_manager.cc:1397] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:50.379515 9947 raft_consensus.cc:348] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.379577 9947 raft_consensus.cc:374] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:50.379595 9947 raft_consensus.cc:729] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be95382fe5594300a624dd27657faa98, State: Initialized, Role: FOLLOWER
I20260619 02:40:50.379637 9947 consensus_queue.cc:260] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.379748 9947 ts_tablet_manager.cc:1428] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:50.380688 10000 tablet_replica.cc:330] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: stopping tablet replica
I20260619 02:40:50.380810 9999 tablet_replica.cc:330] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: stopping tablet replica
I20260619 02:40:50.380829 10000 raft_consensus.cc:2229] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:50.380898 10000 raft_consensus.cc:2258] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:50.380927 9999 raft_consensus.cc:2229] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:50.381031 9999 raft_consensus.cc:2258] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:50.381301 10000 ts_tablet_manager.cc:1905] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:50.381414 9999 ts_tablet_manager.cc:1905] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:50.382723 10000 ts_tablet_manager.cc:1918] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:50.382809 10000 log.cc:1196] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/cf6246a2bb834452b0228589f572b36b
I20260619 02:40:50.383044 10000 ts_tablet_manager.cc:1939] T cf6246a2bb834452b0228589f572b36b P be95382fe5594300a624dd27657faa98: Deleting consensus metadata
I20260619 02:40:50.383469 9424 catalog_manager.cc:4841] TS be95382fe5594300a624dd27657faa98 (127.7.137.3:42461): tablet cf6246a2bb834452b0228589f572b36b (table test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]) successfully deleted
I20260619 02:40:50.384855 9998 tablet_replica.cc:330] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: stopping tablet replica
I20260619 02:40:50.384992 9998 raft_consensus.cc:2229] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:50.385056 9998 raft_consensus.cc:2258] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:50.385385 9998 ts_tablet_manager.cc:1905] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:50.386632 9998 ts_tablet_manager.cc:1918] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:50.386724 9998 log.cc:1196] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/cf6246a2bb834452b0228589f572b36b
I20260619 02:40:50.386971 9998 ts_tablet_manager.cc:1939] T cf6246a2bb834452b0228589f572b36b P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting consensus metadata
I20260619 02:40:50.387573 9423 catalog_manager.cc:4841] TS 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105): tablet cf6246a2bb834452b0228589f572b36b (table test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]) successfully deleted
I20260619 02:40:50.388929 9999 ts_tablet_manager.cc:1918] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:50.389024 9999 log.cc:1196] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/cf6246a2bb834452b0228589f572b36b
I20260619 02:40:50.389303 9999 ts_tablet_manager.cc:1939] T cf6246a2bb834452b0228589f572b36b P 380e5dc39b7b466eac52740fcea0debb: Deleting consensus metadata
I20260619 02:40:50.389751 9423 catalog_manager.cc:4841] TS 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627): tablet cf6246a2bb834452b0228589f572b36b (table test-table [id=3d36bdd1f0a047778b83f2bd9fa01a80]) successfully deleted
I20260619 02:40:50.389998 9945 tablet_bootstrap.cc:492] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: No bootstrap required, opened a new log
I20260619 02:40:50.390064 9945 ts_tablet_manager.cc:1397] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20260619 02:40:50.390236 9945 raft_consensus.cc:348] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.390321 9945 raft_consensus.cc:374] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:50.390349 9945 raft_consensus.cc:729] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Initialized, Role: FOLLOWER
I20260619 02:40:50.390390 9945 consensus_queue.cc:260] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.390482 9945 ts_tablet_manager.cc:1428] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:50.702890 9953 raft_consensus.cc:482] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:50.703016 9953 raft_consensus.cc:504] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.703228 9953 leader_election.cc:288] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627)
I20260619 02:40:50.703538 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36ea32719648ee81d5d1b12db7c749" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb" is_pre_election: true
I20260619 02:40:50.703538 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36ea32719648ee81d5d1b12db7c749" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" is_pre_election: true
I20260619 02:40:50.703642 9736 raft_consensus.cc:2454] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 0.
I20260619 02:40:50.703642 9596 raft_consensus.cc:2454] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 0.
I20260619 02:40:50.703814 9807 leader_election.cc:302] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:50.703943 9953 raft_consensus.cc:2781] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:50.704006 9953 raft_consensus.cc:482] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:50.704021 9953 raft_consensus.cc:3037] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.704739 9953 raft_consensus.cc:504] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.704869 9953 leader_election.cc:288] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 election: Requested vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627)
I20260619 02:40:50.705057 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36ea32719648ee81d5d1b12db7c749" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb"
I20260619 02:40:50.705060 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae36ea32719648ee81d5d1b12db7c749" candidate_uuid: "be95382fe5594300a624dd27657faa98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
I20260619 02:40:50.705137 9596 raft_consensus.cc:3037] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.705137 9736 raft_consensus.cc:3037] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:50.705964 9736 raft_consensus.cc:2454] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 1.
I20260619 02:40:50.705964 9596 raft_consensus.cc:2454] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be95382fe5594300a624dd27657faa98 in term 1.
I20260619 02:40:50.706152 9807 leader_election.cc:302] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:50.706476 9953 raft_consensus.cc:2781] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:50.706553 9953 raft_consensus.cc:686] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 1 LEADER]: Becoming Leader. State: Replica: be95382fe5594300a624dd27657faa98, State: Running, Role: LEADER
I20260619 02:40:50.706610 9953 consensus_queue.cc:237] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:50.707177 9438 catalog_manager.cc:5495] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 reported cstate change: term changed from 0 to 1, leader changed from <none> to be95382fe5594300a624dd27657faa98 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "be95382fe5594300a624dd27657faa98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } health_report { overall_health: HEALTHY } } }
I20260619 02:40:50.736800 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.734533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47260 (local address 127.7.137.3:42461)
0619 02:40:50.734577 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.734579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.734588 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:40:50.734609 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.734609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.734621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.734692 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.734802 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.735363 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.736085 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.736333 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.736391 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.736538 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.736572 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.736613 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.736677 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.736695 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:40:50.740887 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.738924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:52957 (local address 127.7.137.3:42461)
0619 02:40:50.738960 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:50.738961 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:50.738968 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:50.739033 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:50.739033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:50.739040 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:50.739076 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:50.739143 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.739567 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.740230 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.740398 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.740447 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.740613 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.740639 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.740686 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:50.740805 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:40:50.740819 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:40:50.740974 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:50.738861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.7.137.3:52957)
0619 02:40:50.738925 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:50.738936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:50.738978 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:50.739077 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:50.739079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:50.739079 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:50.739109 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.739109 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.739629 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.739630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.740149 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:50.740151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:50.740492 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:50.740493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:50.740536 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:50.740541 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:50.740740 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:50.740743 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:50.740800 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:40:50.740838 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 02:40:50.742808 9596 raft_consensus.cc:1261] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Refusing update from remote peer be95382fe5594300a624dd27657faa98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.742841 9736 raft_consensus.cc:1261] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Refusing update from remote peer be95382fe5594300a624dd27657faa98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:50.743088 9988 consensus_queue.cc:1035] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:50.743218 9988 consensus_queue.cc:1035] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:40:50.749063 9992 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:40:52.271551 9438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:52.271673 9438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:52.272699 9438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Sending DeleteTablet for 3 replicas of tablet ae36ea32719648ee81d5d1b12db7c749
I20260619 02:40:52.272992 9576 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36ea32719648ee81d5d1b12db7c749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39872
I20260619 02:40:52.273021 9716 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36ea32719648ee81d5d1b12db7c749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:55188
I20260619 02:40:52.273115 9839 tablet_service.cc:1514] Processing DeleteTablet for tablet ae36ea32719648ee81d5d1b12db7c749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47238
I20260619 02:40:52.273241 10029 tablet_replica.cc:330] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: stopping tablet replica
I20260619 02:40:52.273303 10029 raft_consensus.cc:2229] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:52.273344 10029 raft_consensus.cc:2258] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.273353 10031 tablet_replica.cc:330] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: stopping tablet replica
I20260619 02:40:52.273411 10031 raft_consensus.cc:2229] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:52.273452 10030 tablet_replica.cc:330] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: stopping tablet replica
I20260619 02:40:52.273496 10031 raft_consensus.cc:2258] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.273502 10030 raft_consensus.cc:2229] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:52.273545 10030 raft_consensus.cc:2258] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.273777 10031 ts_tablet_manager.cc:1905] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.273835 10030 ts_tablet_manager.cc:1905] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.275728 10031 ts_tablet_manager.cc:1918] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:40:52.275728 10030 ts_tablet_manager.cc:1918] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:40:52.275791 10031 log.cc:1196] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/ae36ea32719648ee81d5d1b12db7c749
I20260619 02:40:52.275794 10030 log.cc:1196] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/ae36ea32719648ee81d5d1b12db7c749
I20260619 02:40:52.276048 10030 ts_tablet_manager.cc:1939] T ae36ea32719648ee81d5d1b12db7c749 P 380e5dc39b7b466eac52740fcea0debb: Deleting consensus metadata
I20260619 02:40:52.276074 10031 ts_tablet_manager.cc:1939] T ae36ea32719648ee81d5d1b12db7c749 P be95382fe5594300a624dd27657faa98: Deleting consensus metadata
I20260619 02:40:52.276423 10029 ts_tablet_manager.cc:1905] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.276520 9424 catalog_manager.cc:4841] TS be95382fe5594300a624dd27657faa98 (127.7.137.3:42461): tablet ae36ea32719648ee81d5d1b12db7c749 (table test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]) successfully deleted
I20260619 02:40:52.277626 9423 catalog_manager.cc:4841] TS 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627): tablet ae36ea32719648ee81d5d1b12db7c749 (table test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]) successfully deleted
I20260619 02:40:52.277880 10029 ts_tablet_manager.cc:1918] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:40:52.277941 10029 log.cc:1196] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/ae36ea32719648ee81d5d1b12db7c749
I20260619 02:40:52.278157 10029 ts_tablet_manager.cc:1939] T ae36ea32719648ee81d5d1b12db7c749 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting consensus metadata
I20260619 02:40:52.278596 9423 catalog_manager.cc:4841] TS 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105): tablet ae36ea32719648ee81d5d1b12db7c749 (table test-table [id=a66b94af9d6647bbb3cd6ea2a488963a]) successfully deleted
Jun 19 02:40:52 dist-test-slave-pdnb krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:40:52 dist-test-slave-pdnb krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:52.298429 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.291119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.7.137.62:33879)
0619 02:40:52.291283 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.291285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.291315 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:40:52.291340 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.291341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.291352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.291408 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:52.291944 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.292554 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.293416 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.293619 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.293686 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.295984 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:52.295985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:52.295985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:52.295997 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:52.297502 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:52.297812 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:52.297813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:52.297814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:52.297868 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:52.298026 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:52.298027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:52.298027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:52.298128 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:52.298238 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.298286 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.298356 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260619 02:40:52.300038 9438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:52.300208 9438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:52.301748 9716 tablet_service.cc:1467] Processing CreateTablet for tablet d8b138c237ff4df2bd62d618071da768 (DEFAULT_TABLE table=test-table [id=8f3c4736d2b84421acef20e771a8929f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.301746 9839 tablet_service.cc:1467] Processing CreateTablet for tablet d8b138c237ff4df2bd62d618071da768 (DEFAULT_TABLE table=test-table [id=8f3c4736d2b84421acef20e771a8929f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.301750 9576 tablet_service.cc:1467] Processing CreateTablet for tablet d8b138c237ff4df2bd62d618071da768 (DEFAULT_TABLE table=test-table [id=8f3c4736d2b84421acef20e771a8929f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.301905 9576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b138c237ff4df2bd62d618071da768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.301905 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b138c237ff4df2bd62d618071da768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.301908 9716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b138c237ff4df2bd62d618071da768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.303144 10040 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Bootstrap starting.
I20260619 02:40:52.303344 10042 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Bootstrap starting.
I20260619 02:40:52.303509 10040 tablet_bootstrap.cc:654] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.303694 10042 tablet_bootstrap.cc:654] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.304016 10041 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Bootstrap starting.
I20260619 02:40:52.304195 10040 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: No bootstrap required, opened a new log
I20260619 02:40:52.304303 10040 ts_tablet_manager.cc:1397] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:52.304371 10042 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: No bootstrap required, opened a new log
I20260619 02:40:52.304392 10041 tablet_bootstrap.cc:654] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.304462 10042 ts_tablet_manager.cc:1397] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:52.304471 10040 raft_consensus.cc:348] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.304574 10040 raft_consensus.cc:374] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.304639 10040 raft_consensus.cc:729] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.304620 10042 raft_consensus.cc:348] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.304692 10042 raft_consensus.cc:374] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.304831 10042 raft_consensus.cc:729] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.304806 10040 consensus_queue.cc:260] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.304916 10042 consensus_queue.cc:260] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.304955 10041 tablet_bootstrap.cc:492] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: No bootstrap required, opened a new log
I20260619 02:40:52.304996 10041 ts_tablet_manager.cc:1397] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:52.305017 10040 ts_tablet_manager.cc:1428] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:52.305119 10041 raft_consensus.cc:348] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.305190 10041 raft_consensus.cc:374] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.305191 10042 ts_tablet_manager.cc:1428] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:52.305212 10041 raft_consensus.cc:729] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be95382fe5594300a624dd27657faa98, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.305276 10041 consensus_queue.cc:260] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.305369 10041 ts_tablet_manager.cc:1428] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:52.312513 10047 raft_consensus.cc:482] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:52.312592 10047 raft_consensus.cc:504] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.312973 10047 leader_election.cc:288] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627), be95382fe5594300a624dd27657faa98 (127.7.137.3:42461)
I20260619 02:40:52.316725 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.313153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.7.137.1:59167)
0619 02:40:52.313332 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:52.313342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:52.313403 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:52.313522 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:52.313523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:52.313526 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:52.313596 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.313599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.314438 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.314439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.315684 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.315687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.316438 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.316439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.316517 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.316521 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:52.316569 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:40:52.316634 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260619 02:40:52.316926 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.313183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:55765 (local address 127.7.137.2:33627)
0619 02:40:52.313492 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.313493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.313500 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:52.313520 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.313521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.313529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.313578 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:52.313994 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.314680 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.315934 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.316537 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.316658 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.316671 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.316821 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.316867 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":74,"threads_started":1}
I20260619 02:40:52.316942 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.313122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33627 (local address 127.7.137.1:55765)
0619 02:40:52.313326 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:52.313338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:52.313391 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:52.313582 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:52.313583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:52.313584 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:40:52.313917 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.313919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.314757 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.314758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.315865 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.315867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.316658 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.316659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.316748 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.316752 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:40:52.316809 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:40:52.316851 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":114,"threads_started":1}
I20260619 02:40:52.316973 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.313203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:59167 (local address 127.7.137.3:42461)
0619 02:40:52.313335 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.313338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.313393 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:40:52.313448 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.313449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.313459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.313520 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:40:52.313688 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.314358 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.315789 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.316336 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.316411 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.316425 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.316612 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.316666 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260619 02:40:52.317204 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b138c237ff4df2bd62d618071da768" candidate_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98" is_pre_election: true
I20260619 02:40:52.317409 9876 raft_consensus.cc:2454] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6f011746cc4bfa9bf7e65efeaa9e19 in term 0.
I20260619 02:40:52.317445 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b138c237ff4df2bd62d618071da768" candidate_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb" is_pre_election: true
I20260619 02:40:52.317541 9736 raft_consensus.cc:2454] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6f011746cc4bfa9bf7e65efeaa9e19 in term 0.
I20260619 02:40:52.317590 9528 leader_election.cc:302] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:52.317747 10047 raft_consensus.cc:2781] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:52.317795 10047 raft_consensus.cc:482] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:52.317826 10047 raft_consensus.cc:3037] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.318601 10047 raft_consensus.cc:504] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.318764 10047 leader_election.cc:288] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [CANDIDATE]: Term 1 election: Requested vote from peers 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627), be95382fe5594300a624dd27657faa98 (127.7.137.3:42461)
I20260619 02:40:52.318902 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b138c237ff4df2bd62d618071da768" candidate_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380e5dc39b7b466eac52740fcea0debb"
I20260619 02:40:52.318946 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b138c237ff4df2bd62d618071da768" candidate_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98"
I20260619 02:40:52.319003 9736 raft_consensus.cc:3037] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.319015 9876 raft_consensus.cc:3037] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.319636 9876 raft_consensus.cc:2454] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6f011746cc4bfa9bf7e65efeaa9e19 in term 1.
I20260619 02:40:52.319694 9736 raft_consensus.cc:2454] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6f011746cc4bfa9bf7e65efeaa9e19 in term 1.
I20260619 02:40:52.319814 9528 leader_election.cc:302] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:52.319934 10047 raft_consensus.cc:2781] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:52.320076 10047 raft_consensus.cc:686] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 LEADER]: Becoming Leader. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Running, Role: LEADER
I20260619 02:40:52.320161 10047 consensus_queue.cc:237] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.320874 9438 catalog_manager.cc:5495] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1). New cstate: current_term: 1 leader_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:52.329973 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.327268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.7.137.1:42105)
0619 02:40:52.327418 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.327420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.327426 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:40:52.327462 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.327462 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.327488 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.327575 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:52.327729 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.328232 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.329079 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.329298 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.329381 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.329567 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.329713 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.329807 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.329846 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.329866 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260619 02:40:52.334865 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.332469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:56997 (local address 127.7.137.3:42461)
0619 02:40:52.332526 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.332529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.332562 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:40:52.332649 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.332650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.332661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.332735 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:52.332892 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.333361 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.334131 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.334339 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.334422 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.334605 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.334643 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.334702 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.334761 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.334787 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:40:52.334905 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.332321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42461 (local address 127.7.137.1:56997)
0619 02:40:52.332503 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:52.332515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:52.332578 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:52.332745 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:52.332746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:52.332747 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:52.332835 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.332837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.333446 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.333447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.334048 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.334050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.334424 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.334425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.334497 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.334509 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.334703 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.334706 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:40:52.334758 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:40:52.334799 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260619 02:40:52.337102 9876 raft_consensus.cc:1261] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Refusing update from remote peer 2a6f011746cc4bfa9bf7e65efeaa9e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:52.337109 9736 raft_consensus.cc:1261] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Refusing update from remote peer 2a6f011746cc4bfa9bf7e65efeaa9e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:52.337363 10047 consensus_queue.cc:1035] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:52.337525 10052 consensus_queue.cc:1035] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:52.344460 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.342075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:55737 (local address 127.7.137.1:42105)
0619 02:40:52.342113 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.342115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.342223 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:40:52.342279 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.342280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.342291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.342342 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:52.342456 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.342906 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.343715 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.343985 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.344074 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.344275 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.344314 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.344356 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.344394 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.344412 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:52.344520 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.342018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42105 (local address 127.7.137.3:55737)
0619 02:40:52.342147 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:52.342159 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:52.342228 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:52.342347 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:52.342349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:52.342351 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:40:52.342402 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.342403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.342971 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.342971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.343635 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:52.343637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.344092 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.344093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:52.344152 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.344163 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.344360 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.344361 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:40:52.344393 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 02:40:52.344433 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":85,"threads_started":1}
I20260619 02:40:52.350977 9438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:52.351078 9438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:52.351707 9438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Sending DeleteTablet for 3 replicas of tablet d8b138c237ff4df2bd62d618071da768
I20260619 02:40:52.352036 9576 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b138c237ff4df2bd62d618071da768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39872
I20260619 02:40:52.352056 9716 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b138c237ff4df2bd62d618071da768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:55188
I20260619 02:40:52.352147 10030 tablet_replica.cc:330] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: stopping tablet replica
I20260619 02:40:52.352151 10029 tablet_replica.cc:330] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: stopping tablet replica
I20260619 02:40:52.352200 10029 raft_consensus.cc:2229] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:52.352200 10030 raft_consensus.cc:2229] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:52.352254 10030 raft_consensus.cc:2258] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.352272 10029 raft_consensus.cc:2258] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.352324 9839 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b138c237ff4df2bd62d618071da768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:52 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47238
I20260619 02:40:52.352398 10031 tablet_replica.cc:330] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: stopping tablet replica
I20260619 02:40:52.352444 10031 raft_consensus.cc:2229] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:52.352473 10031 raft_consensus.cc:2258] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:52.352584 10029 ts_tablet_manager.cc:1905] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.352586 10030 ts_tablet_manager.cc:1905] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.352792 10031 ts_tablet_manager.cc:1905] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:52.354133 10030 ts_tablet_manager.cc:1918] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:52.354135 10029 ts_tablet_manager.cc:1918] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:52.354187 10030 log.cc:1196] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/d8b138c237ff4df2bd62d618071da768
I20260619 02:40:52.354187 10029 log.cc:1196] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/d8b138c237ff4df2bd62d618071da768
I20260619 02:40:52.354298 10031 ts_tablet_manager.cc:1918] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:40:52.354348 10031 log.cc:1196] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/d8b138c237ff4df2bd62d618071da768
I20260619 02:40:52.354410 10030 ts_tablet_manager.cc:1939] T d8b138c237ff4df2bd62d618071da768 P 380e5dc39b7b466eac52740fcea0debb: Deleting consensus metadata
I20260619 02:40:52.354410 10029 ts_tablet_manager.cc:1939] T d8b138c237ff4df2bd62d618071da768 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Deleting consensus metadata
I20260619 02:40:52.354560 10031 ts_tablet_manager.cc:1939] T d8b138c237ff4df2bd62d618071da768 P be95382fe5594300a624dd27657faa98: Deleting consensus metadata
I20260619 02:40:52.354884 9423 catalog_manager.cc:4841] TS 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105): tablet d8b138c237ff4df2bd62d618071da768 (table test-table [id=8f3c4736d2b84421acef20e771a8929f]) successfully deleted
I20260619 02:40:52.354959 9423 catalog_manager.cc:4841] TS 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627): tablet d8b138c237ff4df2bd62d618071da768 (table test-table [id=8f3c4736d2b84421acef20e771a8929f]) successfully deleted
I20260619 02:40:52.354990 9438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:40:52.355167 9438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:40:52.355177 9424 catalog_manager.cc:4841] TS be95382fe5594300a624dd27657faa98 (127.7.137.3:42461): tablet d8b138c237ff4df2bd62d618071da768 (table test-table [id=8f3c4736d2b84421acef20e771a8929f]) successfully deleted
I20260619 02:40:52.356407 9576 tablet_service.cc:1467] Processing CreateTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 (DEFAULT_TABLE table=test-table [id=3bdd240c8d584fb3a397ce6905847caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.356437 9716 tablet_service.cc:1467] Processing CreateTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 (DEFAULT_TABLE table=test-table [id=3bdd240c8d584fb3a397ce6905847caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.356531 9576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c8e28bfe324ca7b134d79cd3987a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.356570 9716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c8e28bfe324ca7b134d79cd3987a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.357497 10040 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Bootstrap starting.
I20260619 02:40:52.357749 10042 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Bootstrap starting.
I20260619 02:40:52.357821 10040 tablet_bootstrap.cc:654] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.358129 10042 tablet_bootstrap.cc:654] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.358453 10040 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: No bootstrap required, opened a new log
I20260619 02:40:52.358489 9839 tablet_service.cc:1467] Processing CreateTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 (DEFAULT_TABLE table=test-table [id=3bdd240c8d584fb3a397ce6905847caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:40:52.358501 10040 ts_tablet_manager.cc:1397] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:52.358589 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c8e28bfe324ca7b134d79cd3987a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:52.358621 10040 raft_consensus.cc:348] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.358701 10040 raft_consensus.cc:374] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.358728 10040 raft_consensus.cc:729] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.358774 10040 consensus_queue.cc:260] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.358810 10042 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19: No bootstrap required, opened a new log
I20260619 02:40:52.358850 10042 ts_tablet_manager.cc:1397] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:52.358870 10040 ts_tablet_manager.cc:1428] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:52.359031 10042 raft_consensus.cc:348] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.359138 10042 raft_consensus.cc:374] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.359165 10042 raft_consensus.cc:729] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f011746cc4bfa9bf7e65efeaa9e19, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.359202 10042 consensus_queue.cc:260] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.359338 10042 ts_tablet_manager.cc:1428] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:52.359856 10041 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Bootstrap starting.
I20260619 02:40:52.360176 10041 tablet_bootstrap.cc:654] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:52.360769 10041 tablet_bootstrap.cc:492] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: No bootstrap required, opened a new log
I20260619 02:40:52.360821 10041 ts_tablet_manager.cc:1397] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:40:52.360916 10041 raft_consensus.cc:348] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.360984 10041 raft_consensus.cc:374] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:52.361006 10041 raft_consensus.cc:729] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be95382fe5594300a624dd27657faa98, State: Initialized, Role: FOLLOWER
I20260619 02:40:52.361050 10041 consensus_queue.cc:260] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.361160 10041 ts_tablet_manager.cc:1428] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:52.428484 10046 raft_consensus.cc:482] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:40:52.428606 10046 raft_consensus.cc:504] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.428834 10046 leader_election.cc:288] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), be95382fe5594300a624dd27657faa98 (127.7.137.3:42461)
I20260619 02:40:52.429075 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c8e28bfe324ca7b134d79cd3987a50" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98" is_pre_election: true
I20260619 02:40:52.429080 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c8e28bfe324ca7b134d79cd3987a50" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" is_pre_election: true
I20260619 02:40:52.429171 9596 raft_consensus.cc:2454] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 0.
I20260619 02:40:52.429193 9876 raft_consensus.cc:2454] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 0.
I20260619 02:40:52.429376 9668 leader_election.cc:302] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380e5dc39b7b466eac52740fcea0debb, be95382fe5594300a624dd27657faa98; no voters:
I20260619 02:40:52.429468 10046 raft_consensus.cc:2781] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:40:52.429495 10046 raft_consensus.cc:482] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:40:52.429514 10046 raft_consensus.cc:3037] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.430524 10046 raft_consensus.cc:504] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.430665 10046 leader_election.cc:288] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 election: Requested vote from peers 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105), be95382fe5594300a624dd27657faa98 (127.7.137.3:42461)
I20260619 02:40:52.430837 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c8e28bfe324ca7b134d79cd3987a50" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19"
I20260619 02:40:52.430917 9596 raft_consensus.cc:3037] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.430867 9876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c8e28bfe324ca7b134d79cd3987a50" candidate_uuid: "380e5dc39b7b466eac52740fcea0debb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be95382fe5594300a624dd27657faa98"
I20260619 02:40:52.430963 9876 raft_consensus.cc:3037] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:52.431675 9876 raft_consensus.cc:2454] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 1.
I20260619 02:40:52.431699 9596 raft_consensus.cc:2454] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380e5dc39b7b466eac52740fcea0debb in term 1.
I20260619 02:40:52.431856 9667 leader_election.cc:302] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6f011746cc4bfa9bf7e65efeaa9e19, 380e5dc39b7b466eac52740fcea0debb; no voters:
I20260619 02:40:52.431984 10046 raft_consensus.cc:2781] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:52.432116 10046 raft_consensus.cc:686] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 1 LEADER]: Becoming Leader. State: Replica: 380e5dc39b7b466eac52740fcea0debb, State: Running, Role: LEADER
I20260619 02:40:52.432170 10046 consensus_queue.cc:237] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } }
I20260619 02:40:52.432871 9438 catalog_manager.cc:5495] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb reported cstate change: term changed from 0 to 1, leader changed from <none> to 380e5dc39b7b466eac52740fcea0debb (127.7.137.2). New cstate: current_term: 1 leader_uuid: "380e5dc39b7b466eac52740fcea0debb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380e5dc39b7b466eac52740fcea0debb" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 } health_report { overall_health: UNKNOWN } } }
I20260619 02:40:52.454973 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:52.452324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.7.137.2:33627)
0619 02:40:52.452502 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:52.452504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:52.452511 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:40:52.452557 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:52.452558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:52.452572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:52.452626 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:40:52.452820 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.453530 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.454242 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:52.454542 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:52.454611 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:52.454741 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:40:52.454773 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:40:52.454822 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:52.454872 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 02:40:52.454892 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260619 02:40:52.457224 9876 raft_consensus.cc:1261] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Refusing update from remote peer 380e5dc39b7b466eac52740fcea0debb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:52.457271 9596 raft_consensus.cc:1261] T 88c8e28bfe324ca7b134d79cd3987a50 P 2a6f011746cc4bfa9bf7e65efeaa9e19 [term 1 FOLLOWER]: Refusing update from remote peer 380e5dc39b7b466eac52740fcea0debb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:40:52.457476 10068 consensus_queue.cc:1035] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Connected to new peer: Peer: permanent_uuid: "be95382fe5594300a624dd27657faa98" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:40:52.457572 10046 consensus_queue.cc:1035] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6f011746cc4bfa9bf7e65efeaa9e19" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:40:52.462455 10012 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:40:53.982968 9438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:53.983084 9438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:40:53.983980 9438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b1537b687d406293920134bcd3f430: Sending DeleteTablet for 3 replicas of tablet 88c8e28bfe324ca7b134d79cd3987a50
I20260619 02:40:53.984280 9855 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:53 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47238
I20260619 02:40:53.984272 9576 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:53 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39872
I20260619 02:40:53.984496 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9511
I20260619 02:40:53.984499 9716 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c8e28bfe324ca7b134d79cd3987a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:40:53 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:55188
I20260619 02:40:53.986392 10099 tablet_replica.cc:330] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: stopping tablet replica
I20260619 02:40:53.986464 10099 raft_consensus.cc:2229] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:40:53.986510 10099 raft_consensus.cc:2258] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:53.986616 10098 tablet_replica.cc:330] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: stopping tablet replica
I20260619 02:40:53.986675 10098 raft_consensus.cc:2229] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:40:53.986717 10098 raft_consensus.cc:2258] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:40:53.986810 10099 ts_tablet_manager.cc:1905] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:53.986969 10098 ts_tablet_manager.cc:1905] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:40:53.988417 10099 ts_tablet_manager.cc:1918] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:40:53.988430 10098 ts_tablet_manager.cc:1918] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:40:53.988484 10099 log.cc:1196] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/88c8e28bfe324ca7b134d79cd3987a50
I20260619 02:40:53.988484 10098 log.cc:1196] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/88c8e28bfe324ca7b134d79cd3987a50
I20260619 02:40:53.988715 10098 ts_tablet_manager.cc:1939] T 88c8e28bfe324ca7b134d79cd3987a50 P be95382fe5594300a624dd27657faa98: Deleting consensus metadata
I20260619 02:40:53.988767 10099 ts_tablet_manager.cc:1939] T 88c8e28bfe324ca7b134d79cd3987a50 P 380e5dc39b7b466eac52740fcea0debb: Deleting consensus metadata
I20260619 02:40:53.989073 9424 catalog_manager.cc:4841] TS be95382fe5594300a624dd27657faa98 (127.7.137.3:42461): tablet 88c8e28bfe324ca7b134d79cd3987a50 (table test-table [id=3bdd240c8d584fb3a397ce6905847caa]) successfully deleted
I20260619 02:40:53.989096 9423 catalog_manager.cc:4841] TS 380e5dc39b7b466eac52740fcea0debb (127.7.137.2:33627): tablet 88c8e28bfe324ca7b134d79cd3987a50 (table test-table [id=3bdd240c8d584fb3a397ce6905847caa]) successfully deleted
W20260619 02:40:53.992143 9423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:42105 (error 108)
I20260619 02:40:53.992444 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9651
I20260619 02:40:53.992772 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:53.992527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42105 (local address 127.0.0.1:43436)
0619 02:40:53.992637 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:53.992704 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:42105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
W20260619 02:40:53.992991 9423 catalog_manager.cc:4584] TS 2a6f011746cc4bfa9bf7e65efeaa9e19 (127.7.137.1:42105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88c8e28bfe324ca7b134d79cd3987a50: Network error: Client connection negotiation failed: client connection to 127.7.137.1:42105: connect: Connection refused (error 111)
I20260619 02:40:53.999028 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9791
I20260619 02:40:54.004626 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 9407
2026-06-19T02:40:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6150 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:54 dist-test-slave-pdnb krb5kdc[10104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:54 dist-test-slave-pdnb krb5kdc[10104](info): set up 2 sockets
Jun 19 02:40:54 dist-test-slave-pdnb krb5kdc[10104](info): commencing operation
krb5kdc: starting...
W20260619 02:40:55.036562 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:55.151538 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45885
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:55.230466 10117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:55.230696 10117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:55.230734 10117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:55.231873 10117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:55.231935 10117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:55.231961 10117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:55.231989 10117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:55.232013 10117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:55.233502 10117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:55.233762 10117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:55.233973 10117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:55.238651 10125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.238639 10123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:55.238689 10117 server_base.cc:1034] running on GCE node
W20260619 02:40:55.238679 10122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:55.239171 10117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:55.239459 10117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:55.240625 10117 hybrid_clock.cc:648] HybridClock initialized: now 1781836855240610 us; error 31 us; skew 500 ppm
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:55.242987 10117 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:55.243449 10117 webserver.cc:460] Webserver started at http://127.7.137.62:44945/ using document root <none> and password file <none>
I20260619 02:40:55.243686 10117 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:55.243753 10117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:55.243901 10117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:55.244977 10117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "0786ce03210e4bc0a12dbb244a8f1f4a"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "416a12f24f0c494d7ebfe3859252f94f"
server_key_iv: "651aee04ab5648823a1afc6c41a85d7b"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.245327 10117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "0786ce03210e4bc0a12dbb244a8f1f4a"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "416a12f24f0c494d7ebfe3859252f94f"
server_key_iv: "651aee04ab5648823a1afc6c41a85d7b"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.246592 10117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:55.247258 10132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.247428 10117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:55.247516 10117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "0786ce03210e4bc0a12dbb244a8f1f4a"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "416a12f24f0c494d7ebfe3859252f94f"
server_key_iv: "651aee04ab5648823a1afc6c41a85d7b"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.247593 10117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:55.261461 10117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:55.263289 10117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:55.263451 10117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:55.267443 10117 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45885
I20260619 02:40:55.267477 10184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45885 every 8 connection(s)
I20260619 02:40:55.267850 10117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:55.268788 10185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:55.271037 10185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: Bootstrap starting.
I20260619 02:40:55.271564 10185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:55.271792 10185 log.cc:824] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:55.272444 10185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: No bootstrap required, opened a new log
I20260619 02:40:55.274032 10185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } }
I20260619 02:40:55.274176 10185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:55.274199 10185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0786ce03210e4bc0a12dbb244a8f1f4a, State: Initialized, Role: FOLLOWER
I20260619 02:40:55.274335 10185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } }
I20260619 02:40:55.274412 10185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:55.274434 10185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:55.274462 10185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:55.274922 10185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } }
I20260619 02:40:55.275003 10185 leader_election.cc:302] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0786ce03210e4bc0a12dbb244a8f1f4a; no voters:
I20260619 02:40:55.275128 10185 leader_election.cc:288] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:55.275220 10188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:55.275368 10188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [term 1 LEADER]: Becoming Leader. State: Replica: 0786ce03210e4bc0a12dbb244a8f1f4a, State: Running, Role: LEADER
I20260619 02:40:55.275396 10185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:55.275452 10188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } }
I20260619 02:40:55.275708 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10117
I20260619 02:40:55.275779 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:55.275974 7716 external_mini_cluster.cc:1428] Setting key 6b4038d8652663675495c9afb878d365
I20260619 02:40:55.276644 10190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0786ce03210e4bc0a12dbb244a8f1f4a. Latest consensus state: current_term: 1 leader_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } } }
I20260619 02:40:55.276964 10189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0786ce03210e4bc0a12dbb244a8f1f4a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45885 } } }
I20260619 02:40:55.276973 10190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:55.277046 10189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a [sys.catalog]: This master's current role is: LEADER
W20260619 02:40:55.277585 10203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:40:55.277698 10203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:40:55.277803 10207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:55.277961 10207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:55.279491 10207 catalog_manager.cc:1347] Generated new cluster ID: 6e6684f3d64642f69204a450971fb0d3
I20260619 02:40:55.279541 10207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:55.288255 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.276831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.7.137.62:45885)
0619 02:40:55.277219 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:55.277228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:55.277261 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:40:55.277384 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:55.277391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:55.277450 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:55.277597 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:55.278342 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.279636 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.280676 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.281297 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.281395 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.284153 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:55.284157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:55.284158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:55.284170 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:55.286575 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.286967 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.286970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.286971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.287108 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.287312 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.287313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.287313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.287641 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:55.287812 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:55.287882 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:40:55.288025 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":73,"threads_started":1}
I20260619 02:40:55.300107 10207 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:55.300626 10207 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:55.312304 10207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0786ce03210e4bc0a12dbb244a8f1f4a: Generated new TSK 0
I20260619 02:40:55.312530 10207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:55.341699 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:55.420547 10212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:55.420791 10212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:55.420830 10212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:55.421977 10212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:55.422045 10212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:55.422091 10212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:55.423511 10212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:55.423753 10212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:55.423971 10212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:55.428800 10217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.428857 10218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.428769 10220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:55.428989 10212 server_base.cc:1034] running on GCE node
I20260619 02:40:55.429219 10212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:55.429473 10212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:55.430632 10212 hybrid_clock.cc:648] HybridClock initialized: now 1781836855430615 us; error 29 us; skew 500 ppm
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:55.433151 10212 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:55.433570 10212 webserver.cc:460] Webserver started at http://127.7.137.1:38315/ using document root <none> and password file <none>
I20260619 02:40:55.433744 10212 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:55.433784 10212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:55.433869 10212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:55.434813 10212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "81ef65cb908d4e72ad2e4440337d859c"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "0f695123761f1a2b2d24b9f4f0193afb"
server_key_iv: "602e56c1f1c4dc29e77a63e060565836"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.435169 10212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "81ef65cb908d4e72ad2e4440337d859c"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "0f695123761f1a2b2d24b9f4f0193afb"
server_key_iv: "602e56c1f1c4dc29e77a63e060565836"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.436388 10212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:40:55.437136 10227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.437361 10212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:40:55.437456 10212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "81ef65cb908d4e72ad2e4440337d859c"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "0f695123761f1a2b2d24b9f4f0193afb"
server_key_iv: "602e56c1f1c4dc29e77a63e060565836"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.437515 10212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:55.449687 10212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:55.451499 10212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:55.451603 10212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:55.451769 10212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:55.452018 10212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:55.452047 10212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.452068 10212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:55.452080 10212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.457613 10212 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:41009
I20260619 02:40:55.457665 10340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:41009 every 8 connection(s)
I20260619 02:40:55.457970 10212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:55.466470 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10212
I20260619 02:40:55.466593 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:55.466794 7716 external_mini_cluster.cc:1428] Setting key 25437b095c353001070e93deda3310d1
I20260619 02:40:55.467098 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.458679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:46717 (local address 127.7.137.62:45885)
0619 02:40:55.458720 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:55.458722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:55.458927 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:40:55.459451 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:55.459452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:55.459467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:55.459524 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:55.460458 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.461037 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.462040 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.462245 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.462336 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.464436 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:55.464437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:55.464437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:55.464443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:55.465713 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.466064 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.466065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.466065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.466112 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.466352 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.466353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.466353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.466455 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:55.466509 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:55.466879 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 02:40:55.466982 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:40:55.467272 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.458654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45885 (local address 127.7.137.1:46717)
0619 02:40:55.458816 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:55.458849 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:55.459346 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:55.459589 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:55.459593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:55.460169 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:55.460361 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.460366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.461145 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.461146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.461945 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.461948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.462358 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.462359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.462480 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.462976 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:55.462984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:55.464343 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:55.465794 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.465800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.465802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.466002 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.466169 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.466170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.466171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.466270 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.466541 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:55.466547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:55.466714 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 02:40:55.466841 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 02:40:55.467015 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260619 02:40:55.467696 10341 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45885
I20260619 02:40:55.467788 10341 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:55.467979 10341 heartbeater.cc:511] Master 127.7.137.62:45885 requested a full tablet report, sending...
I20260619 02:40:55.468456 10149 ts_manager.cc:194] Registered new tserver with Master: 81ef65cb908d4e72ad2e4440337d859c (127.7.137.1:41009)
I20260619 02:40:55.469363 10149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:46717
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:55.490784 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:55.568830 10346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:55.569023 10346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:55.569047 10346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:55.570209 10346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:55.570256 10346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:55.570282 10346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:55.571666 10346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:55.571899 10346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:55.572134 10346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:55.577018 10352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.577033 10351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.577023 10354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:55.577261 10346 server_base.cc:1034] running on GCE node
I20260619 02:40:55.577445 10346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:55.577625 10346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:55.578778 10346 hybrid_clock.cc:648] HybridClock initialized: now 1781836855578765 us; error 27 us; skew 500 ppm
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:55.581287 10346 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:55.581715 10346 webserver.cc:460] Webserver started at http://127.7.137.2:33741/ using document root <none> and password file <none>
I20260619 02:40:55.581943 10346 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:55.582007 10346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:55.582157 10346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:55.583176 10346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "734581d20c9c40e499b6d32985f0da02"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "29be113cefd1f2a2b06c6f268a7e74e4"
server_key_iv: "f49267fc10d2cee3880d6dc74eb11457"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.583523 10346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "734581d20c9c40e499b6d32985f0da02"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "29be113cefd1f2a2b06c6f268a7e74e4"
server_key_iv: "f49267fc10d2cee3880d6dc74eb11457"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.584831 10346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:40:55.585543 10361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.585718 10346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:55.585814 10346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "734581d20c9c40e499b6d32985f0da02"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "29be113cefd1f2a2b06c6f268a7e74e4"
server_key_iv: "f49267fc10d2cee3880d6dc74eb11457"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.585901 10346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:55.596189 10346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:55.598423 10346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:55.598577 10346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:55.598802 10346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:55.599088 10346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:55.599146 10346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.599177 10346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:55.599206 10346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.604598 10346 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:36063
I20260619 02:40:55.604681 10474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:36063 every 8 connection(s)
I20260619 02:40:55.605046 10346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:55.613471 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.605726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:49143 (local address 127.7.137.62:45885)
0619 02:40:55.605769 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:55.605771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:55.605987 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:40:55.606481 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:55.606483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:55.606494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:55.606560 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:55.607476 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.607963 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.609056 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.609268 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.609329 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.611196 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:55.611198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:55.611198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:55.611204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:55.612345 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.612685 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.612686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.612686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.612777 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.612984 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.612985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.612985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.613080 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:55.613129 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:55.613348 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:40:55.613408 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:40:55.613616 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.605689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45885 (local address 127.7.137.2:49143)
0619 02:40:55.605852 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:55.605891 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:55.606358 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:55.606632 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:55.606635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:55.607174 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:55.607372 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.607376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.608041 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.608041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.608986 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.608988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.609342 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.609343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.609438 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.609840 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:55.609847 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:55.611116 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:55.612413 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.612418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.612419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.612618 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.612836 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.612837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.612837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.612929 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.613131 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:55.613134 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:55.613244 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 02:40:55.613322 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:40:55.613431 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260619 02:40:55.614099 10475 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45885
I20260619 02:40:55.614197 10475 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:55.614380 10475 heartbeater.cc:511] Master 127.7.137.62:45885 requested a full tablet report, sending...
I20260619 02:40:55.614804 10149 ts_manager.cc:194] Registered new tserver with Master: 734581d20c9c40e499b6d32985f0da02 (127.7.137.2:36063)
I20260619 02:40:55.615129 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10346
I20260619 02:40:55.615242 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:55.615322 10149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:49143
I20260619 02:40:55.615444 7716 external_mini_cluster.cc:1428] Setting key 03943b16c5fbd8889a46450ca0545ece
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:55.638900 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:55.718983 10480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:55.719208 10480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:55.719230 10480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:55.720300 10480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:55.720335 10480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:55.720360 10480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:55.721733 10480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:55.721913 10480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:55.722086 10480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:40:55.727275 10480 server_base.cc:1034] running on GCE node
W20260619 02:40:55.727209 10485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.727197 10486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:55.727202 10488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:55.727625 10480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:55.727830 10480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:55.728950 10480 hybrid_clock.cc:648] HybridClock initialized: now 1781836855728927 us; error 40 us; skew 500 ppm
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:55.731321 10480 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:55.731739 10480 webserver.cc:460] Webserver started at http://127.7.137.3:41921/ using document root <none> and password file <none>
I20260619 02:40:55.731979 10480 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:55.732048 10480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:55.732196 10480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:55.733302 10480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "ca9324b9deec46aa970585349c0fd617"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "876c9c6cdd487472b3117b89a3424fe9"
server_key_iv: "00c83cfbe61f5106de04ad54b80c32f6"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.733649 10480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "ca9324b9deec46aa970585349c0fd617"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "876c9c6cdd487472b3117b89a3424fe9"
server_key_iv: "00c83cfbe61f5106de04ad54b80c32f6"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.734984 10480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:55.735710 10495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.735888 10480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:55.735981 10480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "ca9324b9deec46aa970585349c0fd617"
format_stamp: "Formatted at 2026-06-19 02:40:55 on dist-test-slave-pdnb"
server_key: "876c9c6cdd487472b3117b89a3424fe9"
server_key_iv: "00c83cfbe61f5106de04ad54b80c32f6"
server_key_version: "encryptionkey@0"
I20260619 02:40:55.736059 10480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:55.757165 10480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:55.759177 10480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:55.759364 10480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:55.759603 10480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:55.759922 10480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:55.759976 10480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.760016 10480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:55.760046 10480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:55.765777 10480 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:38955
I20260619 02:40:55.765827 10608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:38955 every 8 connection(s)
I20260619 02:40:55.766176 10480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:55.773936 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10480
I20260619 02:40:55.774039 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:55.774309 7716 external_mini_cluster.cc:1428] Setting key ad46b646f7625e58993b51a3896865c3
I20260619 02:40:55.776070 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.766965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:59071 (local address 127.7.137.62:45885)
0619 02:40:55.767029 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:55.767031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:55.767270 (+ 239us) server_negotiation.cc:407] Connection header received
0619 02:40:55.767733 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:55.767734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:55.767743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:55.767796 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:55.768712 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.769258 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.770319 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.770524 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.770601 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.772941 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:55.772942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:55.772942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:55.772950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:55.774570 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.775070 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.775071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.775072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.775136 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:55.775434 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:55.775435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:55.775435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:55.775583 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:55.775636 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:55.775927 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 02:40:55.776008 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:40:55.776324 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:55.766937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45885 (local address 127.7.137.3:59071)
0619 02:40:55.767092 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:55.767175 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:55.767650 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:55.767856 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:55.767859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:55.768431 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:55.768631 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.768636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.769354 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.769355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.770202 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:55.770204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:55.770619 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:55.770621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:55.770740 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:55.771312 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:55.771320 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:55.772680 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:55.774642 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.774650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.774652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.774992 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.775198 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:55.775199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:55.775199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:55.775374 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:55.775644 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:55.775648 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:55.775778 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 02:40:55.775883 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 02:40:55.776071 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 02:40:55.776870 10609 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45885
I20260619 02:40:55.776953 10609 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:55.777186 10609 heartbeater.cc:511] Master 127.7.137.62:45885 requested a full tablet report, sending...
I20260619 02:40:55.777591 10149 ts_manager.cc:194] Registered new tserver with Master: ca9324b9deec46aa970585349c0fd617 (127.7.137.3:38955)
I20260619 02:40:55.777980 10149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:59071
I20260619 02:40:55.786083 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:40:55.797091 10204 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:40:55.796035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35420 (local address 127.7.137.62:45885)
0619 02:40:55.796119 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:55.796121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:55.796216 (+ 95us) server_negotiation.cc:407] Connection header received
0619 02:40:55.796339 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:55.796340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:55.796350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:55.796419 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:55.797015 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35420: BlockingRecv error: recv got EOF from 127.0.0.1:35420 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:40:55.797925 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10212
I20260619 02:40:55.802428 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10346
I20260619 02:40:55.806514 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10480
I20260619 02:40:55.810708 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10117
2026-06-19T02:40:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1804 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10621](info): set up 2 sockets
Jun 19 02:40:55 dist-test-slave-pdnb krb5kdc[10621](info): commencing operation
krb5kdc: starting...
W20260619 02:40:56.848639 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:56 dist-test-slave-pdnb krb5kdc[10621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:56.963567 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41079
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:57.044010 10634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:57.044241 10634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:57.044277 10634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:57.045451 10634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:57.045516 10634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:57.045540 10634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:57.045562 10634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:57.045616 10634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:57.047084 10634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:57.047338 10634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:57.047552 10634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:57.052248 10642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:57.052170 10640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:57.052171 10639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:57.052397 10634 server_base.cc:1034] running on GCE node
I20260619 02:40:57.052758 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:57.053066 10634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:57.054224 10634 hybrid_clock.cc:648] HybridClock initialized: now 1781836857054212 us; error 29 us; skew 500 ppm
Jun 19 02:40:57 dist-test-slave-pdnb krb5kdc[10621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:57.062284 10634 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:57.062786 10634 webserver.cc:460] Webserver started at http://127.7.137.62:35727/ using document root <none> and password file <none>
I20260619 02:40:57.063025 10634 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:57.063094 10634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:57.063247 10634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:57.064368 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "a448bd21b5d94ca4b838521a7c7932a8"
format_stamp: "Formatted at 2026-06-19 02:40:57 on dist-test-slave-pdnb"
server_key: "da6fdafd8aa5e17c8590226998c76a5a"
server_key_iv: "9b4cd7feca4924fd6039db2fcf39f17f"
server_key_version: "encryptionkey@0"
I20260619 02:40:57.064786 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "a448bd21b5d94ca4b838521a7c7932a8"
format_stamp: "Formatted at 2026-06-19 02:40:57 on dist-test-slave-pdnb"
server_key: "da6fdafd8aa5e17c8590226998c76a5a"
server_key_iv: "9b4cd7feca4924fd6039db2fcf39f17f"
server_key_version: "encryptionkey@0"
I20260619 02:40:57.066078 10634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:40:57.066787 10649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:57.066957 10634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:57.067056 10634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "a448bd21b5d94ca4b838521a7c7932a8"
format_stamp: "Formatted at 2026-06-19 02:40:57 on dist-test-slave-pdnb"
server_key: "da6fdafd8aa5e17c8590226998c76a5a"
server_key_iv: "9b4cd7feca4924fd6039db2fcf39f17f"
server_key_version: "encryptionkey@0"
I20260619 02:40:57.067142 10634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:57.078634 10634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:57.080472 10634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:57.080631 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:57.084843 10634 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41079
I20260619 02:40:57.084903 10701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41079 every 8 connection(s)
I20260619 02:40:57.085258 10634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:57.086102 10702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:57.087987 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10634
I20260619 02:40:57.088089 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:57.088268 7716 external_mini_cluster.cc:1428] Setting key f045f0d7a08fcb56afba0843b2ed4070
I20260619 02:40:57.088466 10702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8: Bootstrap starting.
I20260619 02:40:57.089179 10702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:57.089483 10702 log.cc:824] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:57.090111 10702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8: No bootstrap required, opened a new log
I20260619 02:40:57.092351 10702 raft_consensus.cc:348] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } }
I20260619 02:40:57.092505 10702 raft_consensus.cc:374] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:57.092566 10702 raft_consensus.cc:729] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a448bd21b5d94ca4b838521a7c7932a8, State: Initialized, Role: FOLLOWER
I20260619 02:40:57.092756 10702 consensus_queue.cc:260] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } }
I20260619 02:40:57.092834 10702 raft_consensus.cc:388] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:57.092857 10702 raft_consensus.cc:482] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:57.092921 10702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:57.093429 10702 raft_consensus.cc:504] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } }
I20260619 02:40:57.093801 10702 leader_election.cc:302] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a448bd21b5d94ca4b838521a7c7932a8; no voters:
I20260619 02:40:57.093989 10702 leader_election.cc:288] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:57.094012 10707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:57.094125 10707 raft_consensus.cc:686] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [term 1 LEADER]: Becoming Leader. State: Replica: a448bd21b5d94ca4b838521a7c7932a8, State: Running, Role: LEADER
I20260619 02:40:57.094215 10707 consensus_queue.cc:237] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } }
I20260619 02:40:57.094251 10702 sys_catalog.cc:564] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:57.094521 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a448bd21b5d94ca4b838521a7c7932a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } } }
I20260619 02:40:57.094676 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:57.094870 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a448bd21b5d94ca4b838521a7c7932a8. Latest consensus state: current_term: 1 leader_uuid: "a448bd21b5d94ca4b838521a7c7932a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a448bd21b5d94ca4b838521a7c7932a8" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41079 } } }
I20260619 02:40:57.094993 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:57.095352 10714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 02:40:57 dist-test-slave-pdnb krb5kdc[10621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:57.095844 10714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:57.097635 10714 catalog_manager.cc:1347] Generated new cluster ID: 6044075f06ed4dfab69df7447485f54e
I20260619 02:40:57.097687 10714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:40:57.099609 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:57.088785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.7.137.62:41079)
0619 02:40:57.088999 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:57.089009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:57.089042 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:40:57.089505 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:57.089513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:57.089589 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:57.089742 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:57.090436 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:57.091783 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:57.092540 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:57.093163 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:57.093297 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:57.096164 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:57.096168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:57.096169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:57.096182 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:57.098078 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:57.098463 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:57.098466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:57.098467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:57.098555 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:57.098765 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:57.098766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:57.098766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:57.099003 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:57.099178 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:57.099267 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:40:57.099404 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260619 02:40:57.100493 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:40:57.113243 10714 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:57.113924 10714 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:57.117607 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:57.114014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.7.137.62:41079)
0619 02:40:57.114057 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:57.114059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:57.114244 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:40:57.114316 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:57.114317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:57.114333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:57.114400 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:57.114794 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:57.115319 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:57.116085 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:57.116306 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:57.116387 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:57.117148 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:57.117150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:57.117150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:40:57.117340 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:57.117356 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:40:57.117362 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:57.117426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:57.117489 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:40:57.117516 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:40:57.119987 10714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a448bd21b5d94ca4b838521a7c7932a8: Generated new TSK 0
I20260619 02:40:57.120152 10714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:40:57.123309 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10634
2026-06-19T02:40:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1310 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:57 dist-test-slave-pdnb krb5kdc[10735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:57 dist-test-slave-pdnb krb5kdc[10735](info): set up 2 sockets
Jun 19 02:40:57 dist-test-slave-pdnb krb5kdc[10735](info): commencing operation
krb5kdc: starting...
W20260619 02:40:58.144663 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:40:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:40:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:40:58.259737 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36709
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:58.340346 10748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:58.340593 10748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:58.340633 10748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:58.342353 10748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:40:58.342397 10748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:58.342411 10748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:58.342424 10748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:40:58.342437 10748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:40:58.343822 10748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:58.344012 10748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:58.344183 10748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:58.348964 10756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.348994 10753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.349031 10754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.349068 10748 server_base.cc:1034] running on GCE node
I20260619 02:40:58.349499 10748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:58.349757 10748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:58.350903 10748 hybrid_clock.cc:648] HybridClock initialized: now 1781836858350890 us; error 29 us; skew 500 ppm
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:58.353374 10748 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:40:58.353855 10748 webserver.cc:460] Webserver started at http://127.7.137.62:36789/ using document root <none> and password file <none>
I20260619 02:40:58.354089 10748 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:58.354158 10748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:58.354329 10748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:58.355439 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "59e474b2c35c47f89fd0edf631bd476a"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "4d7ce365001550c95dbbfb1cb6d14dd4"
server_key_iv: "a15c151017448d8a331ba97f9f1b4c03"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.355825 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "59e474b2c35c47f89fd0edf631bd476a"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "4d7ce365001550c95dbbfb1cb6d14dd4"
server_key_iv: "a15c151017448d8a331ba97f9f1b4c03"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.357250 10748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:40:58.357980 10763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.358191 10748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:58.358304 10748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "59e474b2c35c47f89fd0edf631bd476a"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "4d7ce365001550c95dbbfb1cb6d14dd4"
server_key_iv: "a15c151017448d8a331ba97f9f1b4c03"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.358377 10748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:58.369303 10748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:58.371060 10748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:58.371223 10748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:58.375341 10748 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36709
I20260619 02:40:58.375386 10815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36709 every 8 connection(s)
I20260619 02:40:58.375737 10748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:40:58.376693 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:40:58.378922 10816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a: Bootstrap starting.
I20260619 02:40:58.379489 10816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a: Neither blocks nor log segments found. Creating new log.
I20260619 02:40:58.379765 10816 log.cc:824] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:40:58.380446 10816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a: No bootstrap required, opened a new log
I20260619 02:40:58.382037 10816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } }
I20260619 02:40:58.382171 10816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:40:58.382203 10816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e474b2c35c47f89fd0edf631bd476a, State: Initialized, Role: FOLLOWER
I20260619 02:40:58.382365 10816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } }
I20260619 02:40:58.382452 10816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:40:58.382475 10816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:40:58.382503 10816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:40:58.383298 10816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } }
I20260619 02:40:58.383405 10816 leader_election.cc:302] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59e474b2c35c47f89fd0edf631bd476a; no voters:
I20260619 02:40:58.383582 10816 leader_election.cc:288] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:40:58.383656 10819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:40:58.383821 10816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:40:58.383827 10819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [term 1 LEADER]: Becoming Leader. State: Replica: 59e474b2c35c47f89fd0edf631bd476a, State: Running, Role: LEADER
I20260619 02:40:58.383937 10819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } }
I20260619 02:40:58.384322 10821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59e474b2c35c47f89fd0edf631bd476a. Latest consensus state: current_term: 1 leader_uuid: "59e474b2c35c47f89fd0edf631bd476a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } } }
I20260619 02:40:58.384402 10821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:58.384586 10820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59e474b2c35c47f89fd0edf631bd476a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e474b2c35c47f89fd0edf631bd476a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36709 } } }
I20260619 02:40:58.384800 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10748
I20260619 02:40:58.384907 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:40:58.384894 10820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a [sys.catalog]: This master's current role is: LEADER
I20260619 02:40:58.384892 10828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:40:58.385090 7716 external_mini_cluster.cc:1428] Setting key 6756c94f2a3f7ae37791d1369cfb67fe
I20260619 02:40:58.385532 10828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:40:58.388049 10828 catalog_manager.cc:1347] Generated new cluster ID: d31499cf52e845cfa0a679de99485b76
I20260619 02:40:58.388103 10828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:58.396025 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.385514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.7.137.62:36709)
0619 02:40:58.385836 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:58.385845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:58.385874 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:40:58.385983 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:58.385989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:58.386044 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:58.386176 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:58.386852 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.387893 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.388985 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.389599 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.389958 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.392472 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:58.392476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:58.392476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:58.392488 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:58.394424 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.394831 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.394833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.394834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.394929 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.395137 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.395138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.395139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.395413 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:58.395561 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:58.395687 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:40:58.395811 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":77,"threads_started":1}
I20260619 02:40:58.400167 10828 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:40:58.400878 10828 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:40:58.407748 10828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59e474b2c35c47f89fd0edf631bd476a: Generated new TSK 0
I20260619 02:40:58.407919 10828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:40:58.432749 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:58.511714 10842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:58.511947 10842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:58.511986 10842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:58.513208 10842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:58.513268 10842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:58.513314 10842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:40:58.514719 10842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:58.514968 10842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:58.515260 10842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:58.520922 10848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.520942 10850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.520898 10842 server_base.cc:1034] running on GCE node
W20260619 02:40:58.520890 10847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.521312 10842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:58.521559 10842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:58.522729 10842 hybrid_clock.cc:648] HybridClock initialized: now 1781836858522726 us; error 36 us; skew 500 ppm
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:58.525117 10842 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:40:58.525537 10842 webserver.cc:460] Webserver started at http://127.7.137.1:38779/ using document root <none> and password file <none>
I20260619 02:40:58.525766 10842 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:58.525835 10842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:58.525986 10842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:58.527380 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "dcfa847ef2284f7e8f918ad35eaed0ee"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "9391ddd90a95302178b791336cbff15f"
server_key_iv: "9b6d1b0421e8888394341ba682a6c694"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.527745 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "dcfa847ef2284f7e8f918ad35eaed0ee"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "9391ddd90a95302178b791336cbff15f"
server_key_iv: "9b6d1b0421e8888394341ba682a6c694"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.529058 10842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:58.529721 10857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.529881 10842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:58.529975 10842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "dcfa847ef2284f7e8f918ad35eaed0ee"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "9391ddd90a95302178b791336cbff15f"
server_key_iv: "9b6d1b0421e8888394341ba682a6c694"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.530054 10842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:58.561995 10842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:58.563938 10842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:58.564109 10842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:58.564355 10842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:58.564667 10842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:58.564726 10842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.564795 10842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:58.564819 10842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.570665 10842 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:37671
I20260619 02:40:58.570714 10970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:37671 every 8 connection(s)
I20260619 02:40:58.571033 10842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:58.577802 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10842
I20260619 02:40:58.577909 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:40:58.578150 7716 external_mini_cluster.cc:1428] Setting key b9bbf7f320bf1a0b529dbb194695db75
I20260619 02:40:58.580505 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.571728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:58263 (local address 127.7.137.62:36709)
0619 02:40:58.571807 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:58.571810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:58.572024 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:40:58.572490 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:58.572492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:58.572501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:58.572549 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:58.573545 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.574044 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.575005 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.575200 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.575253 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.577416 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:58.577418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:58.577418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:58.577425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:58.578967 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.579473 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.579474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.579474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.579522 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.579789 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.579790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.579790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.579892 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:58.579945 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:58.580286 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 02:40:58.580382 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:40:58.580612 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.571720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36709 (local address 127.7.137.1:58263)
0619 02:40:58.571895 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:58.571932 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:58.572402 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:58.572609 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:58.572626 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:58.573249 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:58.573441 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.573447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.574128 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.574129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.574928 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.574930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.575275 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.575276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.575377 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.575917 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:58.575926 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:58.577334 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:58.579043 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.579050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.579052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.579400 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.579584 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.579585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.579585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.579723 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.579956 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:58.579961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:58.580129 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 02:40:58.580248 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:40:58.580424 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260619 02:40:58.581111 10971 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36709
I20260619 02:40:58.581228 10971 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:58.581399 10971 heartbeater.cc:511] Master 127.7.137.62:36709 requested a full tablet report, sending...
I20260619 02:40:58.581990 10780 ts_manager.cc:194] Registered new tserver with Master: dcfa847ef2284f7e8f918ad35eaed0ee (127.7.137.1:37671)
I20260619 02:40:58.582902 10780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:58263
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:40:58.601917 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:58.681191 10976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:58.681401 10976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:58.681423 10976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:58.682588 10976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:58.682631 10976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:58.682658 10976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:40:58.684298 10976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:58.684542 10976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:58.684819 10976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:58.690605 10981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.690605 10982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.690907 10984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.691298 10976 server_base.cc:1034] running on GCE node
I20260619 02:40:58.691514 10976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:58.691790 10976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:58.692941 10976 hybrid_clock.cc:648] HybridClock initialized: now 1781836858692915 us; error 44 us; skew 500 ppm
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:58.695624 10976 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:40:58.696077 10976 webserver.cc:460] Webserver started at http://127.7.137.2:43607/ using document root <none> and password file <none>
I20260619 02:40:58.696309 10976 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:58.696359 10976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:58.696542 10976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:58.697659 10976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "33d7470532f044058a2e5b1c23ffd3d7"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "2c39544866b26b6ccc34be35fbfe57be"
server_key_iv: "2e1606226c3eecfb50f4934279c44775"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.698020 10976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "33d7470532f044058a2e5b1c23ffd3d7"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "2c39544866b26b6ccc34be35fbfe57be"
server_key_iv: "2e1606226c3eecfb50f4934279c44775"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.699314 10976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:40:58.700065 10991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.700251 10976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:40:58.700333 10976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "33d7470532f044058a2e5b1c23ffd3d7"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "2c39544866b26b6ccc34be35fbfe57be"
server_key_iv: "2e1606226c3eecfb50f4934279c44775"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.700387 10976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:58.717139 10976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:58.719180 10976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:58.719290 10976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:58.719483 10976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:58.719745 10976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:58.719775 10976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.719798 10976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:58.719810 10976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.725610 10976 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:35347
I20260619 02:40:58.725651 11104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:35347 every 8 connection(s)
I20260619 02:40:58.725976 10976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:40:58.726271 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 10976
I20260619 02:40:58.726370 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:40:58.726614 7716 external_mini_cluster.cc:1428] Setting key 06137e624c984146e61e941fd1d47d94
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:40:58.735508 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.726718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:38393 (local address 127.7.137.62:36709)
0619 02:40:58.726767 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:58.726769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:58.727005 (+ 236us) server_negotiation.cc:407] Connection header received
0619 02:40:58.727605 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:58.727606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:58.727619 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:58.727660 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:58.728595 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.729160 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.730151 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.730367 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.730452 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.732545 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:58.732547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:58.732547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:58.732555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:58.734061 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.734471 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.734472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.734473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.734534 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.734783 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.734784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.734784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.734901 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:58.734973 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:58.735332 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 02:40:58.735415 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:40:58.735697 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.726705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36709 (local address 127.7.137.2:38393)
0619 02:40:58.726876 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:58.726925 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:58.727503 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:58.727733 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:58.727737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:58.728308 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:58.728494 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.728499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.729258 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.729259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.730061 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.730064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.730460 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.730461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.730587 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.731102 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:58.731110 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:58.732427 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:58.734144 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.734151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.734153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.734379 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.734602 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.734602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.734603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.734713 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.734973 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:58.734980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:58.735168 (+ 188us) client_negotiation.cc:769] Sending connection context
0619 02:40:58.735311 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:40:58.735470 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260619 02:40:58.736234 11105 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
I20260619 02:40:58.736373 11105 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:58.736577 11105 heartbeater.cc:511] Master 127.7.137.62:36709 requested a full tablet report, sending...
I20260619 02:40:58.737007 10780 ts_manager.cc:194] Registered new tserver with Master: 33d7470532f044058a2e5b1c23ffd3d7 (127.7.137.2:35347)
I20260619 02:40:58.737424 10780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:38393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:40:58.750949 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:40:58.831110 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:40:58.831341 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:40:58.831374 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:40:58.832517 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:40:58.832582 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:40:58.832634 11110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:40:58.834163 11110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:40:58.834437 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:40:58.834662 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:40:58.839583 11118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:40:58.839593 11116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.839808 11110 server_base.cc:1034] running on GCE node
W20260619 02:40:58.839610 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:40:58.840099 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:40:58.840308 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:40:58.841444 11110 hybrid_clock.cc:648] HybridClock initialized: now 1781836858841435 us; error 33 us; skew 500 ppm
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:40:58.843906 11110 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:40:58.844337 11110 webserver.cc:460] Webserver started at http://127.7.137.3:40917/ using document root <none> and password file <none>
I20260619 02:40:58.844573 11110 fs_manager.cc:359] Metadata directory not provided
I20260619 02:40:58.844638 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:40:58.844810 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:40:58.845857 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "442e3c2d10bb4455b36294a874627a0c"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "3352324d8a4b6ff2e4b6107d9aebf014"
server_key_iv: "6a8d203ed036800b41185a4a502951eb"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.846221 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "442e3c2d10bb4455b36294a874627a0c"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "3352324d8a4b6ff2e4b6107d9aebf014"
server_key_iv: "6a8d203ed036800b41185a4a502951eb"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.847528 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:40:58.848220 11125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.848395 11110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:40:58.848491 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "442e3c2d10bb4455b36294a874627a0c"
format_stamp: "Formatted at 2026-06-19 02:40:58 on dist-test-slave-pdnb"
server_key: "3352324d8a4b6ff2e4b6107d9aebf014"
server_key_iv: "6a8d203ed036800b41185a4a502951eb"
server_key_version: "encryptionkey@0"
I20260619 02:40:58.848573 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:40:58.863655 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:40:58.865963 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:40:58.866124 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:40:58.866354 11110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:40:58.866658 11110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:40:58.866714 11110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.866752 11110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:40:58.866781 11110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:40:58.872355 11110 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:34339
I20260619 02:40:58.872426 11238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:34339 every 8 connection(s)
I20260619 02:40:58.872784 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:40:58.875809 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11110
I20260619 02:40:58.875913 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:40:58.876118 7716 external_mini_cluster.cc:1428] Setting key 19781867a06145d8ce9c3a57b0c1da3e
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:58.881945 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.873493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:41003 (local address 127.7.137.62:36709)
0619 02:40:58.873535 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:58.873537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:58.873721 (+ 184us) server_negotiation.cc:407] Connection header received
0619 02:40:58.874193 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:58.874194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:58.874203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:58.874259 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:58.875103 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.875590 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.877104 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.877321 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.877373 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.879460 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:58.879461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:58.879462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:58.879468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:58.880865 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.881228 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.881229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.881229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.881277 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.881480 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.881480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.881481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.881573 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:58.881621 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:58.881832 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 02:40:58.881884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:40:58.882122 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.873453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36709 (local address 127.7.137.3:41003)
0619 02:40:58.873603 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:40:58.873652 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:40:58.874111 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:40:58.874308 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:40:58.874312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:40:58.874853 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:40:58.875032 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.875036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.875700 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.875701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.877040 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:40:58.877043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.877389 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.877390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:40:58.877492 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.877924 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:40:58.877932 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:40:58.879365 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:40:58.880930 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.880935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.880936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.881164 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.881332 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:40:58.881333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:40:58.881333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:40:58.881429 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:40:58.881638 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:40:58.881642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:40:58.881745 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 02:40:58.881821 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:40:58.881926 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260619 02:40:58.882543 11239 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36709
I20260619 02:40:58.882628 11239 heartbeater.cc:464] Registering TS with master...
I20260619 02:40:58.882761 11239 heartbeater.cc:511] Master 127.7.137.62:36709 requested a full tablet report, sending...
I20260619 02:40:58.883098 10780 ts_manager.cc:194] Registered new tserver with Master: 442e3c2d10bb4455b36294a874627a0c (127.7.137.3:34339)
I20260619 02:40:58.883556 10780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:41003
I20260619 02:40:58.887880 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[10735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:40:58.909902 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:40:58.902527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.7.137.62:36709)
0619 02:40:58.902605 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:40:58.902607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:40:58.902706 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:40:58.902800 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:40:58.902801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:40:58.902812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:40:58.902870 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:40:58.903731 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.904328 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.905149 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:40:58.905347 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:40:58.905450 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:40:58.907604 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:40:58.907605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:40:58.907605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:40:58.907612 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:40:58.908958 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.909271 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.909272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.909272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.909321 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:40:58.909477 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:40:58.909477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:40:58.909477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:40:58.909628 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:40:58.909701 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:40:58.909784 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:40:58.909838 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 02:40:58.910130 10780 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33822
I20260619 02:40:58.910554 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10842
I20260619 02:40:58.915134 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10976
I20260619 02:40:58.919481 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11110
I20260619 02:40:58.923673 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 10748
2026-06-19T02:40:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1801 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[11251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[11251](info): set up 2 sockets
Jun 19 02:40:58 dist-test-slave-pdnb krb5kdc[11251](info): commencing operation
krb5kdc: starting...
W20260619 02:40:59.961174 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:00.079531 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41921
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:00.160574 11264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:00.160784 11264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:00.160805 11264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:00.161911 11264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:00.161945 11264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:00.161959 11264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:00.161971 11264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:00.161984 11264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:00.163381 11264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:00.163584 11264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:00.163754 11264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:00.168632 11272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.168648 11269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.168651 11270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:00.168870 11264 server_base.cc:1034] running on GCE node
I20260619 02:41:00.169093 11264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:00.169381 11264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:00.170531 11264 hybrid_clock.cc:648] HybridClock initialized: now 1781836860170521 us; error 35 us; skew 500 ppm
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:00.172986 11264 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:00.173467 11264 webserver.cc:460] Webserver started at http://127.7.137.62:41219/ using document root <none> and password file <none>
I20260619 02:41:00.173718 11264 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:00.173785 11264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:00.173933 11264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:00.174988 11264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "7bd3bbc7ec904a709baead97caed8477"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "26c9ee0eafae200e905c4bf1047abe5e"
server_key_iv: "95382d13dfa65f6e177bd1245a16ed75"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.175343 11264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "7bd3bbc7ec904a709baead97caed8477"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "26c9ee0eafae200e905c4bf1047abe5e"
server_key_iv: "95382d13dfa65f6e177bd1245a16ed75"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.176649 11264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:00.177402 11279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.177568 11264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:00.177680 11264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "7bd3bbc7ec904a709baead97caed8477"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "26c9ee0eafae200e905c4bf1047abe5e"
server_key_iv: "95382d13dfa65f6e177bd1245a16ed75"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.177754 11264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:00.187706 11264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:00.189592 11264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:00.189756 11264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:00.193960 11264 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41921
I20260619 02:41:00.193992 11331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41921 every 8 connection(s)
I20260619 02:41:00.194481 11264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:00.195020 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11264
I20260619 02:41:00.195102 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:00.195135 11332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:00.195317 7716 external_mini_cluster.cc:1428] Setting key 0ce3c42485840a24ba7661db2e509474
I20260619 02:41:00.198495 11332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477: Bootstrap starting.
I20260619 02:41:00.199198 11332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:00.199496 11332 log.cc:824] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:00.200176 11332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477: No bootstrap required, opened a new log
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:00.202452 11332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } }
I20260619 02:41:00.202630 11332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:00.202657 11332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd3bbc7ec904a709baead97caed8477, State: Initialized, Role: FOLLOWER
I20260619 02:41:00.202811 11332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } }
I20260619 02:41:00.202881 11332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:00.202904 11332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:00.202970 11332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:00.203603 11332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } }
I20260619 02:41:00.203725 11332 leader_election.cc:302] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bd3bbc7ec904a709baead97caed8477; no voters:
I20260619 02:41:00.203912 11332 leader_election.cc:288] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:00.203943 11337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:00.204041 11337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [term 1 LEADER]: Becoming Leader. State: Replica: 7bd3bbc7ec904a709baead97caed8477, State: Running, Role: LEADER
I20260619 02:41:00.204125 11337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } }
I20260619 02:41:00.204180 11332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:00.204452 11337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bd3bbc7ec904a709baead97caed8477. Latest consensus state: current_term: 1 leader_uuid: "7bd3bbc7ec904a709baead97caed8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } } }
I20260619 02:41:00.204522 11337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:00.204479 11338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bd3bbc7ec904a709baead97caed8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd3bbc7ec904a709baead97caed8477" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41921 } } }
I20260619 02:41:00.204583 11338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:00.204742 11342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:00.205277 11342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:00.207333 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.195867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50314 (local address 127.7.137.62:41921)
0619 02:41:00.196057 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.196067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.196100 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:41:00.196762 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.196772 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.196840 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.196996 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.197577 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.198960 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.199768 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.200175 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.200296 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.203098 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.203102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.203102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.203114 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.205503 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.205959 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.205963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.205964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.206074 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.206301 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.206302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.206302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.206608 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.206756 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.206933 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.207099 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260619 02:41:00.209152 11342 catalog_manager.cc:1347] Generated new cluster ID: 824adb064e854b349ced9daa740daa6f
I20260619 02:41:00.209211 11342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:00.218619 11342 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:00.219390 11342 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:00.225196 11342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bd3bbc7ec904a709baead97caed8477: Generated new TSK 0
I20260619 02:41:00.225387 11342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:41:00.256342 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:00.343814 11358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:00.344027 11358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:00.344053 11358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:00.345359 11358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:00.345404 11358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:00.345433 11358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:00.346990 11358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:00.347263 11358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:00.347510 11358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:00.353677 11363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.353673 11366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.353714 11364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:00.353900 11358 server_base.cc:1034] running on GCE node
I20260619 02:41:00.354148 11358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:00.354391 11358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:00.355566 11358 hybrid_clock.cc:648] HybridClock initialized: now 1781836860355552 us; error 28 us; skew 500 ppm
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:00.358426 11358 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:41:00.359107 11358 webserver.cc:460] Webserver started at http://127.7.137.1:45261/ using document root <none> and password file <none>
I20260619 02:41:00.359481 11358 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:00.359547 11358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:00.359745 11358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:00.360935 11358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "7ed4dc86a32f4f44840f3b21e277ad58"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "7cffc6dde19557b93e96583b7dc503a9"
server_key_iv: "6ffc531b861215811ecb97519c6ff463"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.361325 11358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "7ed4dc86a32f4f44840f3b21e277ad58"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "7cffc6dde19557b93e96583b7dc503a9"
server_key_iv: "6ffc531b861215811ecb97519c6ff463"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.362761 11358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:00.363513 11373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.363785 11358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:00.363863 11358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "7ed4dc86a32f4f44840f3b21e277ad58"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "7cffc6dde19557b93e96583b7dc503a9"
server_key_iv: "6ffc531b861215811ecb97519c6ff463"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.363956 11358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:00.381186 11358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:00.383447 11358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:00.383602 11358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:00.383879 11358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:00.384234 11358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:00.384290 11358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.384332 11358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:00.384361 11358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.390158 11358 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:44695
I20260619 02:41:00.390198 11486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:44695 every 8 connection(s)
I20260619 02:41:00.390570 11358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:00.391952 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11358
I20260619 02:41:00.392057 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:00.392258 7716 external_mini_cluster.cc:1428] Setting key 56d5ecf7cbbf7d9314bc721157ef2983
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:00.402272 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.391473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:34841 (local address 127.7.137.62:41921)
0619 02:41:00.391549 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.391551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.391758 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:41:00.393149 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.393150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.393164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.393214 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.394125 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.394755 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.395916 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.396292 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.396376 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.398942 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.398944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.398945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.398955 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.400790 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.401244 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.401245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.401246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.401311 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.401563 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.401564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.401564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.401723 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.401808 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.402070 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.402159 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
I20260619 02:41:00.402365 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.391460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41921 (local address 127.7.137.1:34841)
0619 02:41:00.391618 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:00.391662 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:00.393039 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:00.393298 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:00.393303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:00.393716 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:00.393994 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.394001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.394878 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.394880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.395806 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.395808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.396438 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.396440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.396556 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.397340 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:00.397348 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:00.398831 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:00.400874 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.400881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.400883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.401157 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.401388 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.401389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.401389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.401489 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.401798 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:00.401803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:00.401927 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:41:00.402014 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:41:00.402162 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260619 02:41:00.403034 11487 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41921
I20260619 02:41:00.403129 11487 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:00.403353 11487 heartbeater.cc:511] Master 127.7.137.62:41921 requested a full tablet report, sending...
I20260619 02:41:00.403920 11296 ts_manager.cc:194] Registered new tserver with Master: 7ed4dc86a32f4f44840f3b21e277ad58 (127.7.137.1:44695)
I20260619 02:41:00.404793 11296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:34841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:41:00.420068 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:00.503474 11492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:00.503696 11492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:00.503720 11492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:00.505203 11492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:00.505285 11492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:00.505350 11492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:00.506965 11492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:00.507228 11492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:00.507495 11492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:00.512589 11498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.512638 11497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:00.512701 11492 server_base.cc:1034] running on GCE node
W20260619 02:41:00.512616 11500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:00.513058 11492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:00.513335 11492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:00.514511 11492 hybrid_clock.cc:648] HybridClock initialized: now 1781836860514489 us; error 44 us; skew 500 ppm
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:00.517134 11492 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:41:00.517628 11492 webserver.cc:460] Webserver started at http://127.7.137.2:37907/ using document root <none> and password file <none>
I20260619 02:41:00.517863 11492 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:00.517932 11492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:00.518065 11492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:00.519135 11492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "6dd9fbbd00ed43a5bdeb44f753d39eb2"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "c69fb758e87be808449ec87a858578c0"
server_key_iv: "d3b28fcbfe136ea6fbb79bea48efe937"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.519495 11492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "6dd9fbbd00ed43a5bdeb44f753d39eb2"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "c69fb758e87be808449ec87a858578c0"
server_key_iv: "d3b28fcbfe136ea6fbb79bea48efe937"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.521013 11492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:41:00.521775 11507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.521946 11492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:00.522042 11492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "6dd9fbbd00ed43a5bdeb44f753d39eb2"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "c69fb758e87be808449ec87a858578c0"
server_key_iv: "d3b28fcbfe136ea6fbb79bea48efe937"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.522131 11492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:00.542300 11492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:00.544253 11492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:00.544420 11492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:00.544669 11492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:00.545010 11492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:00.545066 11492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.545102 11492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:00.545145 11492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.551167 11492 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:37243
I20260619 02:41:00.551213 11620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:37243 every 8 connection(s)
I20260619 02:41:00.551528 11492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:00.555583 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11492
I20260619 02:41:00.555716 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:00.555925 7716 external_mini_cluster.cc:1428] Setting key ecb59d72c251c2226eb4e250afaf52ea
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:00.561540 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.552287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:53781 (local address 127.7.137.62:41921)
0619 02:41:00.552354 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.552356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.552583 (+ 227us) server_negotiation.cc:407] Connection header received
0619 02:41:00.553552 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.553553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.553563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.553629 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.554226 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.554736 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.555860 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.556156 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.556247 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.558617 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.558619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.558620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.558630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.560101 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.560499 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.560500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.560501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.560561 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.560875 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.560876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.560877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.560993 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.561085 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.561365 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.561448 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:41:00.561725 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.552263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41921 (local address 127.7.137.2:53781)
0619 02:41:00.552436 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:00.552479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:00.553462 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:00.553699 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:00.553703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:00.553956 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:00.554153 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.554158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.554847 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.554848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.555778 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.555781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.556252 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.556254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.556391 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.556953 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:00.556961 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:00.558508 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:00.560191 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.560197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.560200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.560422 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.560639 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.560639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.560640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.560801 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.561080 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:00.561086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:00.561230 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:41:00.561332 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:41:00.561493 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260619 02:41:00.562425 11621 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41921
I20260619 02:41:00.562541 11621 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:00.562747 11621 heartbeater.cc:511] Master 127.7.137.62:41921 requested a full tablet report, sending...
I20260619 02:41:00.563184 11296 ts_manager.cc:194] Registered new tserver with Master: 6dd9fbbd00ed43a5bdeb44f753d39eb2 (127.7.137.2:37243)
I20260619 02:41:00.563562 11296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:53781
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:41:00.581148 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:00.661937 11626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:00.662178 11626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:00.662214 11626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:00.663381 11626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:00.663450 11626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:00.663494 11626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:00.665109 11626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:00.665365 11626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:00.665650 11626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:00.670935 11632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.670941 11631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:00.670998 11634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:00.671159 11626 server_base.cc:1034] running on GCE node
I20260619 02:41:00.671379 11626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:00.671586 11626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:00.672760 11626 hybrid_clock.cc:648] HybridClock initialized: now 1781836860672710 us; error 28 us; skew 500 ppm
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:00.675318 11626 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:41:00.675804 11626 webserver.cc:460] Webserver started at http://127.7.137.3:44063/ using document root <none> and password file <none>
I20260619 02:41:00.676029 11626 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:00.676098 11626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:00.676262 11626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:00.677397 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "6f5957fb1bce494b96939715738d33c4"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "e22014ea908a65faba50ab0a9e1a0124"
server_key_iv: "b594912d0cad1977f85a8b6317ddaa7d"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.677757 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "6f5957fb1bce494b96939715738d33c4"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "e22014ea908a65faba50ab0a9e1a0124"
server_key_iv: "b594912d0cad1977f85a8b6317ddaa7d"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.679077 11626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:00.679890 11641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.680114 11626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:00.680225 11626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "6f5957fb1bce494b96939715738d33c4"
format_stamp: "Formatted at 2026-06-19 02:41:00 on dist-test-slave-pdnb"
server_key: "e22014ea908a65faba50ab0a9e1a0124"
server_key_iv: "b594912d0cad1977f85a8b6317ddaa7d"
server_key_version: "encryptionkey@0"
I20260619 02:41:00.680310 11626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:00.696444 11626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:00.698891 11626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:00.699061 11626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:00.699321 11626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:00.699642 11626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:00.699694 11626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.699734 11626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:00.699764 11626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:00.705735 11626 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:39341
I20260619 02:41:00.705845 11754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:39341 every 8 connection(s)
I20260619 02:41:00.706151 11626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:00.715555 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.706908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:47467 (local address 127.7.137.62:41921)
0619 02:41:00.706970 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.706973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.707180 (+ 207us) server_negotiation.cc:407] Connection header received
0619 02:41:00.708133 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.708134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.708147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.708194 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.708870 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.709372 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.710387 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.710583 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.710675 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.712884 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.712886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.712886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.712896 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.714155 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.714576 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.714577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.714578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.714636 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.714898 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.714899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.714899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.715017 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.715095 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.715380 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.715459 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:41:00.715749 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.706900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41921 (local address 127.7.137.3:47467)
0619 02:41:00.707054 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:00.707096 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:00.708019 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:00.708264 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:00.708268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:00.708538 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:00.708782 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.708789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.709465 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.709467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.710292 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:00.710294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.710682 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.710683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:00.710815 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.711413 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:00.711422 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:00.712793 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:00.714253 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.714260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.714263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.714487 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.714718 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:00.714719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:00.714719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:00.714826 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:00.715084 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:00.715090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:00.715241 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 02:41:00.715346 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 02:41:00.715505 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260619 02:41:00.716279 11755 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41921
I20260619 02:41:00.716312 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11626
I20260619 02:41:00.716368 11755 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:00.716414 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:00.716568 11755 heartbeater.cc:511] Master 127.7.137.62:41921 requested a full tablet report, sending...
I20260619 02:41:00.716637 7716 external_mini_cluster.cc:1428] Setting key c80a3ec0baa04fd0907a8120b4302b0e
I20260619 02:41:00.717027 11295 ts_manager.cc:194] Registered new tserver with Master: 6f5957fb1bce494b96939715738d33c4 (127.7.137.3:39341)
I20260619 02:41:00.717473 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:41:00.717514 11295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:47467
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:41:00.741204 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.732825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.7.137.1:44695)
0619 02:41:00.732995 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.733001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.733014 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:41:00.733043 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.733046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.733067 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.733152 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.733780 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.734540 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.735323 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.735657 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.735750 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.738119 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.738128 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.738128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.738136 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.739836 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.740238 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.740241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.740242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.740304 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.740527 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.740528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.740528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.740680 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.740877 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.740973 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.741073 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:00.752257 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:00.742710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.7.137.62:41921)
0619 02:41:00.742773 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:00.742779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:00.742788 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:00.742811 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:00.742812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:00.742821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:00.742874 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:00.743560 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.744119 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.745301 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:00.745687 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:00.745784 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:00.748783 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:00.748784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:00.748785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:00.748795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:00.750720 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.751180 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.751182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.751182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.751250 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:00.751491 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:00.751493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:00.751493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:00.751716 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:00.751826 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:00.751995 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:41:00.752102 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 02:41:00.752491 11295 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50330
I20260619 02:41:00.753422 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11358
I20260619 02:41:00.758965 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11492
I20260619 02:41:00.763283 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11626
I20260619 02:41:00.767407 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11264
2026-06-19T02:41:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1844 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11767](info): set up 2 sockets
Jun 19 02:41:00 dist-test-slave-pdnb krb5kdc[11767](info): commencing operation
krb5kdc: starting...
W20260619 02:41:01.820667 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:01 dist-test-slave-pdnb krb5kdc[11767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:01.937224 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38817
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:38817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:02.016834 11780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.017117 11780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.017166 11780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.018357 11780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:02.018429 11780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.018455 11780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.018482 11780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:02.018513 11780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:02.020035 11780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:38817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.020344 11780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.020570 11780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.025346 11786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.025408 11785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.025377 11788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.025692 11780 server_base.cc:1034] running on GCE node
I20260619 02:41:02.025972 11780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.026316 11780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.027486 11780 hybrid_clock.cc:648] HybridClock initialized: now 1781836862027464 us; error 35 us; skew 500 ppm
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:02.030020 11780 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:02.030378 11780 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.030426 11780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.030510 11780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.031473 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "9aa820fa687a4fb99e6af1026893d01d"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "2f53aa944bcb6d8581fbce5f71259165"
server_key_iv: "edd9acff5630579192b28a3830c489af"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.031764 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "9aa820fa687a4fb99e6af1026893d01d"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "2f53aa944bcb6d8581fbce5f71259165"
server_key_iv: "edd9acff5630579192b28a3830c489af"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.032979 11780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:41:02.033679 11794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.033843 11780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.033911 11780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "9aa820fa687a4fb99e6af1026893d01d"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "2f53aa944bcb6d8581fbce5f71259165"
server_key_iv: "edd9acff5630579192b28a3830c489af"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.033962 11780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:02.060209 11780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:02.066162 11780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:02.066280 11780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:02.070233 11780 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:38817
I20260619 02:41:02.070259 11846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:38817 every 8 connection(s)
I20260619 02:41:02.070581 11780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:02.071520 11847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:02.071707 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11780
I20260619 02:41:02.071815 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:02.072018 7716 external_mini_cluster.cc:1428] Setting key 057980be61e147afabd1e4755b0fbb4f
I20260619 02:41:02.075132 11847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: Bootstrap starting.
I20260619 02:41:02.075817 11847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:02.076102 11847 log.cc:824] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:02.076889 11847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: No bootstrap required, opened a new log
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:02.079084 11847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } }
I20260619 02:41:02.079272 11847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:02.079339 11847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa820fa687a4fb99e6af1026893d01d, State: Initialized, Role: FOLLOWER
I20260619 02:41:02.079468 11847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } }
I20260619 02:41:02.079525 11847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:02.079578 11847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:02.079625 11847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:02.080294 11847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } }
I20260619 02:41:02.080421 11847 leader_election.cc:302] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aa820fa687a4fb99e6af1026893d01d; no voters:
I20260619 02:41:02.080633 11847 leader_election.cc:288] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:02.080680 11852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:02.080839 11852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [term 1 LEADER]: Becoming Leader. State: Replica: 9aa820fa687a4fb99e6af1026893d01d, State: Running, Role: LEADER
I20260619 02:41:02.080952 11847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:02.080937 11852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } }
I20260619 02:41:02.081502 11854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aa820fa687a4fb99e6af1026893d01d. Latest consensus state: current_term: 1 leader_uuid: "9aa820fa687a4fb99e6af1026893d01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } } }
I20260619 02:41:02.081483 11853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aa820fa687a4fb99e6af1026893d01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa820fa687a4fb99e6af1026893d01d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38817 } } }
I20260619 02:41:02.081672 11853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:02.081857 11854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:02.082031 11860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:02.082562 11860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:02.083932 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.072534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.7.137.62:38817)
0619 02:41:02.072709 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.072718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.072770 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:41:02.073411 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.073420 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.073483 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.073638 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.074228 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.075585 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.076417 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.076996 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.077110 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.079734 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.079737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.079737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:02.079748 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.081758 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.082172 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.082175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.082175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.082275 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.082529 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.082530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.082530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.082875 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.083010 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.083576 (+ 566us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.083720 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260619 02:41:02.084488 11860 catalog_manager.cc:1347] Generated new cluster ID: ac989dffc35a4e5388c195f333c63da8
I20260619 02:41:02.084563 11860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:02.117319 11860 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:02.117884 11860 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:02.122588 11860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: Generated new TSK 0
I20260619 02:41:02.122785 11860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:41:02.151541 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:02.246438 11873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.246645 11873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.246668 11873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.247816 11873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.247859 11873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.247932 11873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:02.249461 11873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.249704 11873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.249928 11873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.254808 11881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.254801 11879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.254804 11878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.255162 11873 server_base.cc:1034] running on GCE node
I20260619 02:41:02.255395 11873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.255650 11873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.256799 11873 hybrid_clock.cc:648] HybridClock initialized: now 1781836862256789 us; error 38 us; skew 500 ppm
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:02.259449 11873 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:41:02.259859 11873 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.259936 11873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.260114 11873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.261408 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "ec226d3ad87f41488fd51ff4308c17b1"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "621f0cf8b22787f29123a3f9e83636b4"
server_key_iv: "2b49190b761f0acf31cb96ad43de86a3"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.261809 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "ec226d3ad87f41488fd51ff4308c17b1"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "621f0cf8b22787f29123a3f9e83636b4"
server_key_iv: "2b49190b761f0acf31cb96ad43de86a3"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.263281 11873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:02.263993 11887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.264168 11873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:02.264276 11873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "ec226d3ad87f41488fd51ff4308c17b1"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "621f0cf8b22787f29123a3f9e83636b4"
server_key_iv: "2b49190b761f0acf31cb96ad43de86a3"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.264356 11873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:02.274926 11873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:02.276985 11873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:02.277164 11873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:02.277422 11873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:02.277767 11873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:02.277827 11873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.277874 11873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:02.277911 11873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.284412 11873 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45995
I20260619 02:41:02.284444 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45995 every 8 connection(s)
I20260619 02:41:02.284747 11873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:02.286823 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 11873
I20260619 02:41:02.286931 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:02.287158 7716 external_mini_cluster.cc:1428] Setting key 483526d2980dadd8bb0989d3c21c1c9e
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:02.295133 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.285449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38817 (local address 127.7.137.1:34783)
0619 02:41:02.285605 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.285647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.286111 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.286343 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.286347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.286596 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:02.287334 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.287341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.288210 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.288211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.289485 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.289487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.289985 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.289985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.290078 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.290583 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:02.290591 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:02.291971 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:02.293831 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.293837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.293839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.294046 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.294267 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.294268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.294268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.294361 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.294647 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:02.294651 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:02.294776 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.294855 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.294971 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260619 02:41:02.295138 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.285482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:34783 (local address 127.7.137.62:38817)
0619 02:41:02.285525 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.285528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.285740 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:41:02.286198 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.286199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.286207 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.286269 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.287476 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.288116 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.289571 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.289900 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.289979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.292071 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.292073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.292073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:02.292082 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.293753 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.294120 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.294121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.294121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.294193 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.294431 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.294432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.294432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.294577 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.294655 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.294911 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.295009 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
I20260619 02:41:02.295764 12001 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38817
I20260619 02:41:02.295876 12001 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:02.296021 12001 heartbeater.cc:511] Master 127.7.137.62:38817 requested a full tablet report, sending...
I20260619 02:41:02.296530 11811 ts_manager.cc:194] Registered new tserver with Master: ec226d3ad87f41488fd51ff4308c17b1 (127.7.137.1:45995)
I20260619 02:41:02.297185 11811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:34783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:41:02.311460 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:02.390810 12006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.391053 12006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.391089 12006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.392230 12006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.392292 12006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.392333 12006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:02.393847 12006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.394106 12006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.394330 12006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.399194 12012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.399194 12011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.399240 12006 server_base.cc:1034] running on GCE node
W20260619 02:41:02.399224 12014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.399678 12006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.399924 12006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.401078 12006 hybrid_clock.cc:648] HybridClock initialized: now 1781836862401109 us; error 72 us; skew 500 ppm
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:02.403551 12006 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:41:02.403926 12006 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.403991 12006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.404122 12006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.405203 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "bc8a3b84d25d4d54a50eb437ac20923a"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "e4d5349b84a3f185f039496c42900e66"
server_key_iv: "1e8dd94064d5bf88b77aeebee209904c"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.405557 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "bc8a3b84d25d4d54a50eb437ac20923a"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "e4d5349b84a3f185f039496c42900e66"
server_key_iv: "1e8dd94064d5bf88b77aeebee209904c"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.406813 12006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:02.407524 12020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.407693 12006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:02.407783 12006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "bc8a3b84d25d4d54a50eb437ac20923a"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "e4d5349b84a3f185f039496c42900e66"
server_key_iv: "1e8dd94064d5bf88b77aeebee209904c"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.407862 12006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:02.425158 12006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:02.427377 12006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:02.427531 12006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:02.427763 12006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:02.428071 12006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:02.428121 12006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.428164 12006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:02.428188 12006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.433908 12006 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:33555
I20260619 02:41:02.433945 12133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:33555 every 8 connection(s)
I20260619 02:41:02.434245 12006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:02.436287 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12006
I20260619 02:41:02.436411 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:02.436650 7716 external_mini_cluster.cc:1428] Setting key ceff1eb1ae89dbafda13634668ba244c
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:02.444985 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.434955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:41701 (local address 127.7.137.62:38817)
0619 02:41:02.434995 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.434997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.435227 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:41:02.435665 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.435665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.435674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.435731 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.436934 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.437649 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.439113 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.439428 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.439501 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.441932 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.441933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.441934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:02.441943 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.443582 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.443979 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.443980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.443981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.444041 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.444291 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.444292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.444293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.444428 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.444506 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.444765 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.444855 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:02.445080 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.434921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38817 (local address 127.7.137.2:41701)
0619 02:41:02.435095 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.435136 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.435590 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.435782 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.435785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.436033 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:02.436840 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.436848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.437743 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.437745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.439031 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.439034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.439509 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.439510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.439609 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.440115 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:02.440123 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:02.441829 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:02.443676 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.443681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.443683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.443899 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.444110 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.444111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.444111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.444220 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.444496 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:02.444500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:02.444617 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.444699 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.444882 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:41:02.445622 12134 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38817
I20260619 02:41:02.445719 12134 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:02.445873 12134 heartbeater.cc:511] Master 127.7.137.62:38817 requested a full tablet report, sending...
I20260619 02:41:02.446246 11811 ts_manager.cc:194] Registered new tserver with Master: bc8a3b84d25d4d54a50eb437ac20923a (127.7.137.2:33555)
I20260619 02:41:02.446627 11811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:41701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:41:02.461807 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:02.540508 12139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.540752 12139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.540795 12139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.541956 12139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.542025 12139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.542073 12139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:02.543569 12139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.543833 12139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.544057 12139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.548892 12144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.548890 12147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.548985 12145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.549408 12139 server_base.cc:1034] running on GCE node
I20260619 02:41:02.549737 12139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.549974 12139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.551127 12139 hybrid_clock.cc:648] HybridClock initialized: now 1781836862551113 us; error 29 us; skew 500 ppm
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:02.553555 12139 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:41:02.553917 12139 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.553965 12139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.554054 12139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.555038 12139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "b03bc8a9785f4df1b6c80510b74d2768"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "f203717ab37c4a6bed46455a8f6bf0ef"
server_key_iv: "5eea3cd00df41f2c207a5b9ede9be506"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.555323 12139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "b03bc8a9785f4df1b6c80510b74d2768"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "f203717ab37c4a6bed46455a8f6bf0ef"
server_key_iv: "5eea3cd00df41f2c207a5b9ede9be506"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.556492 12139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:41:02.557291 12153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.557485 12139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:02.557548 12139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "b03bc8a9785f4df1b6c80510b74d2768"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "f203717ab37c4a6bed46455a8f6bf0ef"
server_key_iv: "5eea3cd00df41f2c207a5b9ede9be506"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.557641 12139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:02.587440 12139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:02.590044 12139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:02.590224 12139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:02.590479 12139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:02.590826 12139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:02.590885 12139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.590934 12139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:02.590961 12139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.597318 12139 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45287
I20260619 02:41:02.597389 12266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45287 every 8 connection(s)
I20260619 02:41:02.597694 12139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:41:02 dist-test-slave-pdnb krb5kdc[11767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:02.606487 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12139
I20260619 02:41:02.606567 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:02.606791 7716 external_mini_cluster.cc:1428] Setting key d8295b5099566041c76c6f70a541dac5
I20260619 02:41:02.606891 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.598439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:47101 (local address 127.7.137.62:38817)
0619 02:41:02.598482 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.598484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.598714 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:41:02.599208 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.599208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.599217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.599275 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.600226 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.600706 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.601825 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.602004 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.602055 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.604257 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.604258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.604259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:02.604265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.605498 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.605886 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.605887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.605887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.605934 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.606202 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.606203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.606203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.606345 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.606416 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.606719 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.606795 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:02.607103 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.598421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38817 (local address 127.7.137.3:47101)
0619 02:41:02.598582 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.598626 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.599121 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.599333 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.599336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.599613 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:02.600145 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.600150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.600834 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.600835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.601746 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.601748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.602096 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.602097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.602209 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.602821 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:02.602831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:02.604172 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:02.605593 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.605598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.605600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.605819 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.606011 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:02.606012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:02.606013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:02.606129 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:02.606413 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:02.606417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:02.606572 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.606686 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.606807 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260619 02:41:02.607633 12267 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38817
I20260619 02:41:02.607733 12267 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:02.607878 12267 heartbeater.cc:511] Master 127.7.137.62:38817 requested a full tablet report, sending...
I20260619 02:41:02.608299 11811 ts_manager.cc:194] Registered new tserver with Master: b03bc8a9785f4df1b6c80510b74d2768 (127.7.137.3:45287)
I20260619 02:41:02.608672 11811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:47101
I20260619 02:41:02.618615 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:41:02.626315 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.619968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.7.137.62:38817)
0619 02:41:02.620062 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.620064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.620179 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:41:02.620261 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.620262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.620279 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.620339 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.621037 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.621528 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.622264 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.622542 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.622639 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.623871 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.623873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.623873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:02.623880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.625174 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.625516 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.625517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.625517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.625573 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:02.625773 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:02.625773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:02.625774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:02.625925 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.626025 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.626135 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.626211 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:41:02.627717 11811 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:02.629355 11811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:02.635566 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.631709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.7.137.1:45995)
0619 02:41:02.631894 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.631898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.631906 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:02.631935 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.631937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.631948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.632040 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:02.632281 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633390 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.634714 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635162 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.635255 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.635313 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.635448 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.635486 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260619 02:41:02.635555 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.631523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45995 (local address 127.0.0.1:60984)
0619 02:41:02.631634 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.631674 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.631736 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.632042 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.632045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.632046 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:02.632112 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.632114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633492 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.633493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.634615 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.634616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635265 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.635266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.635343 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.635355 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.635425 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.635478 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":73,"thread_start_us":63,"threads_started":1}
I20260619 02:41:02.635756 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.631742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33555 (local address 127.0.0.1:60748)
0619 02:41:02.631855 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.631864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.631901 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.632555 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.632556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.632558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:02.632613 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.632614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633764 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.633765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.634676 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.634679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635495 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.635496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.635544 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.635547 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.635578 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.635619 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":83,"threads_started":1}
I20260619 02:41:02.635921 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.631525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45287 (local address 127.0.0.1:48908)
0619 02:41:02.631666 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.631676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.631734 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.632171 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.632172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.632173 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:02.632210 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.632211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633264 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.633265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.634584 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.634587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635478 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.635479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.635543 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.635551 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.635591 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.635619 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":92,"threads_started":1}
I20260619 02:41:02.636159 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.631894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.7.137.3:45287)
0619 02:41:02.632028 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.632033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.632043 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:02.632080 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.632081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.632097 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.632175 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:02.632263 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633152 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.634669 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635374 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.635788 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.635926 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.636038 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.636078 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260619 02:41:02.636268 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.632139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.7.137.2:33555)
0619 02:41:02.632386 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.632390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.632399 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:02.632434 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.632436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.632451 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.632551 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:02.632670 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.633666 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.634762 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.635414 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.636045 (+ 631us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.636111 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.636135 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.636176 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":81,"threads_started":1}
I20260619 02:41:02.636366 11935 tablet_service.cc:1467] Processing CreateTablet for tablet afeca2e0adfa4f528a5f5500181cb37d (DEFAULT_TABLE table=test-table [id=1799ec2322604317822e017da52b9d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:02.636788 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afeca2e0adfa4f528a5f5500181cb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:02.636900 12201 tablet_service.cc:1467] Processing CreateTablet for tablet afeca2e0adfa4f528a5f5500181cb37d (DEFAULT_TABLE table=test-table [id=1799ec2322604317822e017da52b9d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:02.637008 12068 tablet_service.cc:1467] Processing CreateTablet for tablet afeca2e0adfa4f528a5f5500181cb37d (DEFAULT_TABLE table=test-table [id=1799ec2322604317822e017da52b9d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:02.637255 12068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afeca2e0adfa4f528a5f5500181cb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:02.637254 12201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afeca2e0adfa4f528a5f5500181cb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:02.639398 12284 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: Bootstrap starting.
I20260619 02:41:02.639998 12284 tablet_bootstrap.cc:654] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:02.640336 12286 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Bootstrap starting.
I20260619 02:41:02.640864 12285 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Bootstrap starting.
I20260619 02:41:02.640929 12284 log.cc:824] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:02.641119 12286 tablet_bootstrap.cc:654] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:02.641479 12285 tablet_bootstrap.cc:654] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:02.641765 12286 log.cc:824] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:02.641798 12284 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: No bootstrap required, opened a new log
I20260619 02:41:02.641852 12284 ts_tablet_manager.cc:1397] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 02:41:02.642073 12285 log.cc:824] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:02.642522 12286 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: No bootstrap required, opened a new log
I20260619 02:41:02.642638 12286 ts_tablet_manager.cc:1397] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:41:02.642704 12285 tablet_bootstrap.cc:492] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: No bootstrap required, opened a new log
I20260619 02:41:02.642755 12285 ts_tablet_manager.cc:1397] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:02.643829 12284 raft_consensus.cc:348] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.643999 12284 raft_consensus.cc:374] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:02.644035 12284 raft_consensus.cc:729] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec226d3ad87f41488fd51ff4308c17b1, State: Initialized, Role: FOLLOWER
I20260619 02:41:02.644171 12284 consensus_queue.cc:260] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.644361 12285 raft_consensus.cc:348] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.644443 12284 ts_tablet_manager.cc:1428] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:41:02.644502 12285 raft_consensus.cc:374] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:02.644544 12285 raft_consensus.cc:729] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8a3b84d25d4d54a50eb437ac20923a, State: Initialized, Role: FOLLOWER
I20260619 02:41:02.644572 12001 heartbeater.cc:503] Master 127.7.137.62:38817 was elected leader, sending a full tablet report...
I20260619 02:41:02.644663 12285 consensus_queue.cc:260] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.644690 12286 raft_consensus.cc:348] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.644869 12286 raft_consensus.cc:374] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:02.644907 12286 raft_consensus.cc:729] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03bc8a9785f4df1b6c80510b74d2768, State: Initialized, Role: FOLLOWER
I20260619 02:41:02.644963 12285 ts_tablet_manager.cc:1428] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:02.645022 12286 consensus_queue.cc:260] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.645066 12134 heartbeater.cc:503] Master 127.7.137.62:38817 was elected leader, sending a full tablet report...
I20260619 02:41:02.645291 12267 heartbeater.cc:503] Master 127.7.137.62:38817 was elected leader, sending a full tablet report...
I20260619 02:41:02.645483 12286 ts_tablet_manager.cc:1428] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260619 02:41:02.684762 12135 tablet.cc:2369] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:02.692838 12292 raft_consensus.cc:482] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:02.692958 12292 raft_consensus.cc:504] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.693415 12292 leader_election.cc:288] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec226d3ad87f41488fd51ff4308c17b1 (127.7.137.1:45995), bc8a3b84d25d4d54a50eb437ac20923a (127.7.137.2:33555)
I20260619 02:41:02.696528 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.693825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:59899 (local address 127.7.137.2:33555)
0619 02:41:02.693887 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.693889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.694067 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:41:02.694219 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.694220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.694229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.694270 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:02.694399 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.694865 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.695804 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.696171 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.696258 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.696267 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.696434 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.696472 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:41:02.696518 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.693614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45995 (local address 127.7.137.3:54475)
0619 02:41:02.693668 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.693682 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.693723 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.693928 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.693929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.693931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:02.694051 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.694054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.694849 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.694850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.695679 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.695681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.696243 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.696244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.696323 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.696327 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.696378 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.696433 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260619 02:41:02.696524 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.693720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:54475 (local address 127.7.137.1:45995)
0619 02:41:02.693782 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.693784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.693792 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:02.693820 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.693821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.693831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.693924 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:02.694167 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.694766 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.695777 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.696158 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.696215 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.696248 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.696410 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.696451 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:41:02.696542 12293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.693737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33555 (local address 127.7.137.3:59899)
0619 02:41:02.693921 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.693932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:02.694122 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:02.694273 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:02.694274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:02.694276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:02.694340 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.694341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.694938 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.694939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.695693 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:02.695694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.696275 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.696276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:02.696343 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.696346 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:02.696420 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:41:02.696456 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":9,"thread_start_us":112,"threads_started":1}
I20260619 02:41:02.696902 12088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afeca2e0adfa4f528a5f5500181cb37d" candidate_uuid: "b03bc8a9785f4df1b6c80510b74d2768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" is_pre_election: true
I20260619 02:41:02.696935 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afeca2e0adfa4f528a5f5500181cb37d" candidate_uuid: "b03bc8a9785f4df1b6c80510b74d2768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec226d3ad87f41488fd51ff4308c17b1" is_pre_election: true
I20260619 02:41:02.697088 12088 raft_consensus.cc:2454] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03bc8a9785f4df1b6c80510b74d2768 in term 0.
I20260619 02:41:02.697100 11955 raft_consensus.cc:2454] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03bc8a9785f4df1b6c80510b74d2768 in term 0.
I20260619 02:41:02.697338 12157 leader_election.cc:302] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b03bc8a9785f4df1b6c80510b74d2768, bc8a3b84d25d4d54a50eb437ac20923a; no voters:
I20260619 02:41:02.697502 12292 raft_consensus.cc:2781] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:02.697574 12292 raft_consensus.cc:482] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:02.697593 12292 raft_consensus.cc:3037] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:02.698328 12292 raft_consensus.cc:504] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.698436 12292 leader_election.cc:288] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [CANDIDATE]: Term 1 election: Requested vote from peers ec226d3ad87f41488fd51ff4308c17b1 (127.7.137.1:45995), bc8a3b84d25d4d54a50eb437ac20923a (127.7.137.2:33555)
I20260619 02:41:02.698632 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afeca2e0adfa4f528a5f5500181cb37d" candidate_uuid: "b03bc8a9785f4df1b6c80510b74d2768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec226d3ad87f41488fd51ff4308c17b1"
I20260619 02:41:02.698705 12088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afeca2e0adfa4f528a5f5500181cb37d" candidate_uuid: "b03bc8a9785f4df1b6c80510b74d2768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8a3b84d25d4d54a50eb437ac20923a"
I20260619 02:41:02.698725 11955 raft_consensus.cc:3037] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:02.698774 12088 raft_consensus.cc:3037] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:02.699661 11955 raft_consensus.cc:2454] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03bc8a9785f4df1b6c80510b74d2768 in term 1.
I20260619 02:41:02.699676 12088 raft_consensus.cc:2454] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03bc8a9785f4df1b6c80510b74d2768 in term 1.
I20260619 02:41:02.699832 12157 leader_election.cc:302] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b03bc8a9785f4df1b6c80510b74d2768, ec226d3ad87f41488fd51ff4308c17b1; no voters:
I20260619 02:41:02.699921 12292 raft_consensus.cc:2781] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:02.700093 12292 raft_consensus.cc:686] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 1 LEADER]: Becoming Leader. State: Replica: b03bc8a9785f4df1b6c80510b74d2768, State: Running, Role: LEADER
I20260619 02:41:02.700203 12292 consensus_queue.cc:237] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } }
I20260619 02:41:02.701095 11810 catalog_manager.cc:5495] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 reported cstate change: term changed from 0 to 1, leader changed from <none> to b03bc8a9785f4df1b6c80510b74d2768 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "b03bc8a9785f4df1b6c80510b74d2768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b03bc8a9785f4df1b6c80510b74d2768" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:02.711954 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.709455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.7.137.3:45287)
0619 02:41:02.709511 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.709513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.709522 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:02.709555 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.709556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.709578 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.709638 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:02.709801 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.710262 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.711070 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:02.711347 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:02.711417 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:02.711601 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:02.711699 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:02.711776 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.711808 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.711826 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:41:02.713740 11955 raft_consensus.cc:1261] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 1 FOLLOWER]: Refusing update from remote peer b03bc8a9785f4df1b6c80510b74d2768: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:02.713740 12088 raft_consensus.cc:1261] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 1 FOLLOWER]: Refusing update from remote peer b03bc8a9785f4df1b6c80510b74d2768: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:02.714064 12292 consensus_queue.cc:1035] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec226d3ad87f41488fd51ff4308c17b1" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:02.714213 12294 consensus_queue.cc:1035] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8a3b84d25d4d54a50eb437ac20923a" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:02.716464 12297 mvcc.cc:205] Tried to move back new op lower bound from 7298403789672853504 to 7298403789620813824. Current Snapshot: MvccSnapshot[applied={T|T < 7298403789672853504}]
I20260619 02:41:02.717103 12298 mvcc.cc:205] Tried to move back new op lower bound from 7298403789672853504 to 7298403789620813824. Current Snapshot: MvccSnapshot[applied={T|T < 7298403789672853504 or (T in {7298403789672853504})}]
I20260619 02:41:02.718690 11811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:02.718798 11811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:02.719556 11811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9aa820fa687a4fb99e6af1026893d01d: Sending DeleteTablet for 3 replicas of tablet afeca2e0adfa4f528a5f5500181cb37d
I20260619 02:41:02.719885 12201 tablet_service.cc:1514] Processing DeleteTablet for tablet afeca2e0adfa4f528a5f5500181cb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:02 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:48908
I20260619 02:41:02.719925 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet afeca2e0adfa4f528a5f5500181cb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:02 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:60984
I20260619 02:41:02.720112 12306 tablet_replica.cc:330] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: stopping tablet replica
I20260619 02:41:02.720100 12068 tablet_service.cc:1514] Processing DeleteTablet for tablet afeca2e0adfa4f528a5f5500181cb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:02 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:60748
I20260619 02:41:02.720198 12306 raft_consensus.cc:2229] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:02.720278 12306 raft_consensus.cc:2258] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:02.720314 12307 tablet_replica.cc:330] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1: stopping tablet replica
I20260619 02:41:02.720319 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11873
I20260619 02:41:02.720405 12307 raft_consensus.cc:2229] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:02.720498 12307 raft_consensus.cc:2258] T afeca2e0adfa4f528a5f5500181cb37d P ec226d3ad87f41488fd51ff4308c17b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:02.721472 12306 ts_tablet_manager.cc:1905] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:02.723300 12306 ts_tablet_manager.cc:1918] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:02.723369 12306 log.cc:1196] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/afeca2e0adfa4f528a5f5500181cb37d
I20260619 02:41:02.723635 12306 ts_tablet_manager.cc:1939] T afeca2e0adfa4f528a5f5500181cb37d P b03bc8a9785f4df1b6c80510b74d2768: Deleting consensus metadata
I20260619 02:41:02.724035 11796 catalog_manager.cc:4841] TS b03bc8a9785f4df1b6c80510b74d2768 (127.7.137.3:45287): tablet afeca2e0adfa4f528a5f5500181cb37d (table test-table [id=1799ec2322604317822e017da52b9d01]) successfully deleted
I20260619 02:41:02.724933 12308 tablet_replica.cc:330] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: stopping tablet replica
I20260619 02:41:02.725073 12308 raft_consensus.cc:2229] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:02.725207 12308 raft_consensus.cc:2258] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:02.725700 12308 ts_tablet_manager.cc:1905] T afeca2e0adfa4f528a5f5500181cb37d P bc8a3b84d25d4d54a50eb437ac20923a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:41:02.726765 11798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:45995 (error 108)
I20260619 02:41:02.727028 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12006
I20260619 02:41:02.732851 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12139
I20260619 02:41:02.732985 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.732838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45995 (local address 127.0.0.1:60986)
0619 02:41:02.732893 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.732939 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:41:02.733068 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.732916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33555 (local address 127.0.0.1:60756)
0619 02:41:02.733001 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:02.733035 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.2:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 02:41:02.733733 11798 catalog_manager.cc:4584] TS ec226d3ad87f41488fd51ff4308c17b1 (127.7.137.1:45995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afeca2e0adfa4f528a5f5500181cb37d: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45995: connect: Connection refused (error 111)
I20260619 02:41:02.737569 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 11780
2026-06-19T02:41:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1970 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T02:41:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:02Z Disabled control of system clock
I20260619 02:41:02.764636 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33117
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:46347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:33117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:41:02.844128 12313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.844360 12313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.844383 12313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.845474 12313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:02.845510 12313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.845523 12313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.845535 12313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:02.845547 12313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:02.846786 12313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:33117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.846978 12313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.847213 12313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.851974 12318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.852099 12313 server_base.cc:1034] running on GCE node
W20260619 02:41:02.851984 12321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.851985 12319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.852473 12313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.852815 12313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.853989 12313 hybrid_clock.cc:648] HybridClock initialized: now 1781836862853976 us; error 25 us; skew 500 ppm
I20260619 02:41:02.855176 12313 webserver.cc:460] Webserver started at http://127.7.137.62:33867/ using document root <none> and password file <none>
I20260619 02:41:02.855430 12313 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.855501 12313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.855662 12313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.856787 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "1d749e00d3254141888e11dfcb0d96e3"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "3e994531679c0d6adc2dfb37c3028266"
server_key_iv: "97c04bace57749a3adb698e214f45a57"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.857200 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "1d749e00d3254141888e11dfcb0d96e3"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "3e994531679c0d6adc2dfb37c3028266"
server_key_iv: "97c04bace57749a3adb698e214f45a57"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.858613 12313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:02.859368 12327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.859542 12313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:02.859644 12313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "1d749e00d3254141888e11dfcb0d96e3"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "3e994531679c0d6adc2dfb37c3028266"
server_key_iv: "97c04bace57749a3adb698e214f45a57"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.859730 12313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:02.863220 12313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:02.863498 12313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:02.863641 12313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:02.867394 12313 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:33117
I20260619 02:41:02.867424 12379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:33117 every 8 connection(s)
I20260619 02:41:02.867756 12313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:02.868398 12380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:02.868666 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12313
I20260619 02:41:02.868808 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:02.869030 7716 external_mini_cluster.cc:1428] Setting key 14b36f1b4db62740f607d11de928a84c
I20260619 02:41:02.871663 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:02.869605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.7.137.62:33117)
0619 02:41:02.869791 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:02.869798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:02.869822 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:41:02.870280 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:02.870287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:02.870325 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:02.870472 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:02.871175 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:02.871179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:02.871179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:02.871183 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:02.871232 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:02.871243 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:02.871317 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:02.871384 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:41:02.871522 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260619 02:41:02.872262 12380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: Bootstrap starting.
I20260619 02:41:02.872897 12380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:02.873167 12380 log.cc:824] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:02.873838 12380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: No bootstrap required, opened a new log
I20260619 02:41:02.875176 12380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } }
I20260619 02:41:02.875329 12380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:02.875373 12380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d749e00d3254141888e11dfcb0d96e3, State: Initialized, Role: FOLLOWER
I20260619 02:41:02.875501 12380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } }
I20260619 02:41:02.875597 12380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:02.875641 12380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:02.875695 12380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:02.876170 12380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } }
I20260619 02:41:02.876286 12380 leader_election.cc:302] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d749e00d3254141888e11dfcb0d96e3; no voters:
I20260619 02:41:02.876459 12380 leader_election.cc:288] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:02.876744 12380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:02.876838 12385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:02.877010 12385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1d749e00d3254141888e11dfcb0d96e3, State: Running, Role: LEADER
I20260619 02:41:02.877239 12385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } }
I20260619 02:41:02.877645 12387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d749e00d3254141888e11dfcb0d96e3. Latest consensus state: current_term: 1 leader_uuid: "1d749e00d3254141888e11dfcb0d96e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } } }
I20260619 02:41:02.877746 12387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:02.877822 12386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d749e00d3254141888e11dfcb0d96e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d749e00d3254141888e11dfcb0d96e3" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33117 } } }
I20260619 02:41:02.877902 12386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3 [sys.catalog]: This master's current role is: LEADER
W20260619 02:41:02.878538 12400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:41:02.878631 12400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:41:02.878695 12398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:02.878826 12398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:02.880301 12398 catalog_manager.cc:1347] Generated new cluster ID: d1298c006c14415f977cfc91dbc38860
I20260619 02:41:02.880360 12398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:02.889286 12398 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:02.889457 12398 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:02.892635 12398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: Generated new TSK 0
I20260619 02:41:02.892832 12398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:02.895514 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--builtin_ntp_servers=127.7.137.20:46347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:41:02.978463 12404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:02.978662 12404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:02.978683 12404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:02.979825 12404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:02.979893 12404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:02.979934 12404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:02.981281 12404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:02.981551 12404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:02.981850 12404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:02.987780 12410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.987890 12412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:02.987780 12409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:02.988020 12404 server_base.cc:1034] running on GCE node
I20260619 02:41:02.988243 12404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:02.988512 12404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:02.989696 12404 hybrid_clock.cc:648] HybridClock initialized: now 1781836862989678 us; error 32 us; skew 500 ppm
I20260619 02:41:02.991132 12404 webserver.cc:460] Webserver started at http://127.7.137.1:33289/ using document root <none> and password file <none>
I20260619 02:41:02.991392 12404 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:02.991446 12404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:02.991568 12404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:02.992960 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "99d117cdef6e440bba8bf34541167c2c"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "704f28d581379d52da21055aa904ae0f"
server_key_iv: "8220df583debc1fd91b9d680dd8d5a53"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.993335 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "99d117cdef6e440bba8bf34541167c2c"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "704f28d581379d52da21055aa904ae0f"
server_key_iv: "8220df583debc1fd91b9d680dd8d5a53"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.994902 12404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:02.995824 12418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:02.996054 12404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:02.996142 12404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "99d117cdef6e440bba8bf34541167c2c"
format_stamp: "Formatted at 2026-06-19 02:41:02 on dist-test-slave-pdnb"
server_key: "704f28d581379d52da21055aa904ae0f"
server_key_iv: "8220df583debc1fd91b9d680dd8d5a53"
server_key_version: "encryptionkey@0"
I20260619 02:41:02.996214 12404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.000926 12404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.001236 12404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.001365 12404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.001575 12404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:03.001904 12404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:03.001955 12404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.001991 12404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:03.002023 12404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.009232 12404 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:33703
I20260619 02:41:03.009266 12531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:33703 every 8 connection(s)
I20260619 02:41:03.009636 12404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:03.010355 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12404
I20260619 02:41:03.010450 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:03.010629 7716 external_mini_cluster.cc:1428] Setting key 5a6502ffab1db778f00b2f70832e8425
I20260619 02:41:03.011564 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--builtin_ntp_servers=127.7.137.20:46347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:41:03.013077 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.010366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:35639 (local address 127.7.137.62:33117)
0619 02:41:03.010410 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.010413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.010723 (+ 310us) server_negotiation.cc:407] Connection header received
0619 02:41:03.011350 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.011351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.011357 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.011406 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.012761 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.012763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.012763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.012769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.012784 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.012788 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.012840 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.012968 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.012985 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:41:03.013669 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.010387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33117 (local address 127.7.137.1:35639)
0619 02:41:03.010583 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.010622 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.011259 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.011481 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.011485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.011495 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.012572 (+ 1077us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.012573 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.012600 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.012629 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.012867 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.012873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.012877 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.013361 (+ 484us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.013432 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260619 02:41:03.014261 12532 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33117
I20260619 02:41:03.014345 12532 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:03.014482 12532 heartbeater.cc:511] Master 127.7.137.62:33117 requested a full tablet report, sending...
I20260619 02:41:03.015003 12344 ts_manager.cc:194] Registered new tserver with Master: 99d117cdef6e440bba8bf34541167c2c (127.7.137.1:33703)
W20260619 02:41:03.094640 12535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:03.094872 12535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:03.094910 12535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:03.096079 12535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:03.096149 12535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:03.096194 12535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:03.097671 12535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:03.097947 12535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:03.098243 12535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:03.103439 12540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.103513 12543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.103559 12541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.103533 12535 server_base.cc:1034] running on GCE node
I20260619 02:41:03.103817 12535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:03.104064 12535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:03.105230 12535 hybrid_clock.cc:648] HybridClock initialized: now 1781836863105222 us; error 46 us; skew 500 ppm
I20260619 02:41:03.106551 12535 webserver.cc:460] Webserver started at http://127.7.137.2:42765/ using document root <none> and password file <none>
I20260619 02:41:03.106817 12535 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:03.106891 12535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:03.107034 12535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:03.108100 12535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "bc3a23512b474d49892ce9def0de4167"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "ced3d4322b681f8920e0736f8ed5e694"
server_key_iv: "8d29624b85bac151a3e3c39b051e0e2f"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.108464 12535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "bc3a23512b474d49892ce9def0de4167"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "ced3d4322b681f8920e0736f8ed5e694"
server_key_iv: "8d29624b85bac151a3e3c39b051e0e2f"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.109848 12535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:03.110598 12549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.110785 12535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:03.110880 12535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "bc3a23512b474d49892ce9def0de4167"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "ced3d4322b681f8920e0736f8ed5e694"
server_key_iv: "8d29624b85bac151a3e3c39b051e0e2f"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.110967 12535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.114492 12535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.114763 12535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.114902 12535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.115093 12535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:03.115382 12535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:03.115435 12535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.115474 12535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:03.115501 12535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.121263 12535 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:34463
I20260619 02:41:03.121301 12662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:34463 every 8 connection(s)
I20260619 02:41:03.121642 12535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:03.124359 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.122336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:39055 (local address 127.7.137.62:33117)
0619 02:41:03.122375 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.122377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.122623 (+ 246us) server_negotiation.cc:407] Connection header received
0619 02:41:03.123049 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.123050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.123054 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.123112 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.124145 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.124146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.124146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.124150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.124162 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.124166 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.124215 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.124293 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.124305 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:03.124528 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.122318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33117 (local address 127.7.137.2:39055)
0619 02:41:03.122487 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.122526 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.122969 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.123163 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.123166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.123174 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.124008 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.124008 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.124035 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.124064 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.124227 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.124231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.124233 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.124289 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.124336 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260619 02:41:03.125134 12663 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33117
I20260619 02:41:03.125236 12663 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:03.125396 12663 heartbeater.cc:511] Master 127.7.137.62:33117 requested a full tablet report, sending...
I20260619 02:41:03.125761 12344 ts_manager.cc:194] Registered new tserver with Master: bc3a23512b474d49892ce9def0de4167 (127.7.137.2:34463)
I20260619 02:41:03.126881 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12535
I20260619 02:41:03.126984 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:03.127156 7716 external_mini_cluster.cc:1428] Setting key e4f9fe18014235a30aca5945a4ffccbe
I20260619 02:41:03.127988 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--builtin_ntp_servers=127.7.137.20:46347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:41:03.207610 12666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:03.207818 12666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:03.207840 12666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:03.209007 12666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:03.209046 12666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:03.209071 12666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:03.210403 12666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:33117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:03.210654 12666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:03.210949 12666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:03.216120 12672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.216100 12674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.216104 12671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.216132 12666 server_base.cc:1034] running on GCE node
I20260619 02:41:03.216536 12666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:03.216841 12666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:03.218003 12666 hybrid_clock.cc:648] HybridClock initialized: now 1781836863217993 us; error 37 us; skew 500 ppm
I20260619 02:41:03.219290 12666 webserver.cc:460] Webserver started at http://127.7.137.3:34011/ using document root <none> and password file <none>
I20260619 02:41:03.219525 12666 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:03.219594 12666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:03.219749 12666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:03.220872 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "7a2576c4ecf142aaa8a87da872d027c7"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "0146738fea4657d1b725baf5ecf9d7b1"
server_key_iv: "5d6b2cfe9bfdc3f13dd163a99a94c3c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.221249 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "7a2576c4ecf142aaa8a87da872d027c7"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "0146738fea4657d1b725baf5ecf9d7b1"
server_key_iv: "5d6b2cfe9bfdc3f13dd163a99a94c3c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.222508 12666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:03.223208 12680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.223368 12666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:03.223464 12666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "7a2576c4ecf142aaa8a87da872d027c7"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "0146738fea4657d1b725baf5ecf9d7b1"
server_key_iv: "5d6b2cfe9bfdc3f13dd163a99a94c3c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.223542 12666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.226941 12666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.227190 12666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.227324 12666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.227519 12666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:03.227867 12666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:03.227931 12666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.227968 12666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:03.227996 12666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.233560 12666 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:34827
I20260619 02:41:03.233600 12793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:34827 every 8 connection(s)
I20260619 02:41:03.233927 12666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:03.236606 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.234634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:34727 (local address 127.7.137.62:33117)
0619 02:41:03.234686 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.234689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.234905 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:41:03.235318 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.235319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.235323 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.235374 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.236414 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.236415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.236415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.236419 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.236432 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.236436 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.236481 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.236546 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.236557 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:03.236847 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.234627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:33117 (local address 127.7.137.3:34727)
0619 02:41:03.234795 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.234826 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.235242 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.235423 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.235426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.235433 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.236281 (+ 848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.236282 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.236307 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.236333 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.236493 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.236497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.236499 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.236569 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.236614 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260619 02:41:03.237286 12794 heartbeater.cc:347] Connected to a master server at 127.7.137.62:33117
I20260619 02:41:03.237370 12794 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:03.237506 12794 heartbeater.cc:511] Master 127.7.137.62:33117 requested a full tablet report, sending...
I20260619 02:41:03.237838 12344 ts_manager.cc:194] Registered new tserver with Master: 7a2576c4ecf142aaa8a87da872d027c7 (127.7.137.3:34827)
I20260619 02:41:03.242425 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12666
I20260619 02:41:03.242502 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:03.242697 7716 external_mini_cluster.cc:1428] Setting key 2b6c59a5c06c7dfb9d0f90dfc6d3fd9b
I20260619 02:41:03.243511 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:41:03.245716 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.244711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.7.137.62:33117)
0619 02:41:03.244792 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.244795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.244898 (+ 103us) server_negotiation.cc:407] Connection header received
0619 02:41:03.244935 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.244935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.244939 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.244983 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.245480 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.245481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.245481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.245485 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.245496 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.245499 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.245537 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.245588 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.245598 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:41:03.247251 12344 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:03.248786 12344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:03.252458 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.250760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:34827 (local address 127.0.0.1:42088)
0619 02:41:03.250927 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.250968 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.251022 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.251398 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.251405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.251413 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.251987 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.251988 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.252004 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.252026 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.252269 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.252275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.252278 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.252343 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.252356 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260619 02:41:03.252492 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.251056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.7.137.1:33703)
0619 02:41:03.251287 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.251292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.251333 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:41:03.251419 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.251428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.251436 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.251498 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.252221 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.252227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.252228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.252233 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.252264 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.252287 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.252342 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.252394 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.252409 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":160,"threads_started":1}
I20260619 02:41:03.252521 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.250963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33703 (local address 127.0.0.1:60340)
0619 02:41:03.251279 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.251290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.251339 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.251740 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.251741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.251743 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.252114 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.252115 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.252123 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.252134 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.252337 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.252338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.252339 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.252389 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.252403 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":62,"threads_started":1}
I20260619 02:41:03.253180 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.250782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:34463 (local address 127.0.0.1:54974)
0619 02:41:03.251066 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.251076 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.251120 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.251483 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.251484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.251486 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.252245 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.252246 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.252252 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.252267 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.253082 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.253082 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.253083 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.253130 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.253140 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":88,"threads_started":1}
I20260619 02:41:03.253235 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.250867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.7.137.2:34463)
0619 02:41:03.251338 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.251341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.251350 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:03.251375 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.251377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.251386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.252156 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.252938 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.252941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.252942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.252945 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.252976 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.252989 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.253055 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.253135 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.253149 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":81,"threads_started":1}
I20260619 02:41:03.253261 12466 tablet_service.cc:1467] Processing CreateTablet for tablet cf9b793966d84df39c4784410519a1af (DEFAULT_TABLE table=test-table [id=a9e368a289774f1b89cf77e57e81c708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:03.253655 12466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf9b793966d84df39c4784410519a1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.253930 12597 tablet_service.cc:1467] Processing CreateTablet for tablet cf9b793966d84df39c4784410519a1af (DEFAULT_TABLE table=test-table [id=a9e368a289774f1b89cf77e57e81c708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:03.254220 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf9b793966d84df39c4784410519a1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.256258 12808 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: Bootstrap starting.
I20260619 02:41:03.256354 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.251120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.7.137.3:34827)
0619 02:41:03.251262 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.251267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.251277 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:03.251304 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.251306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.251315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.251380 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.252103 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.252110 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.252111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.252115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.252153 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.252177 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.252269 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.252754 (+ 485us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.252770 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260619 02:41:03.256598 12809 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Bootstrap starting.
I20260619 02:41:03.256987 12808 tablet_bootstrap.cc:654] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.257045 12728 tablet_service.cc:1467] Processing CreateTablet for tablet cf9b793966d84df39c4784410519a1af (DEFAULT_TABLE table=test-table [id=a9e368a289774f1b89cf77e57e81c708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:03.257287 12809 tablet_bootstrap.cc:654] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.257362 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf9b793966d84df39c4784410519a1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.257351 12808 log.cc:824] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.257613 12809 log.cc:824] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.258067 12808 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: No bootstrap required, opened a new log
I20260619 02:41:03.258119 12808 ts_tablet_manager.cc:1397] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:03.259895 12812 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Bootstrap starting.
I20260619 02:41:03.260310 12808 raft_consensus.cc:348] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.260486 12808 raft_consensus.cc:374] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.260514 12812 tablet_bootstrap.cc:654] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.260528 12808 raft_consensus.cc:729] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d117cdef6e440bba8bf34541167c2c, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.260655 12808 consensus_queue.cc:260] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.260864 12812 log.cc:824] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.260985 12808 ts_tablet_manager.cc:1428] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:03.260995 12532 heartbeater.cc:503] Master 127.7.137.62:33117 was elected leader, sending a full tablet report...
I20260619 02:41:03.261318 12809 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: No bootstrap required, opened a new log
I20260619 02:41:03.261384 12809 ts_tablet_manager.cc:1397] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 02:41:03.261610 12812 tablet_bootstrap.cc:492] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: No bootstrap required, opened a new log
I20260619 02:41:03.261675 12812 ts_tablet_manager.cc:1397] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:03.263581 12809 raft_consensus.cc:348] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.263724 12809 raft_consensus.cc:374] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.263768 12809 raft_consensus.cc:729] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc3a23512b474d49892ce9def0de4167, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.263703 12812 raft_consensus.cc:348] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.263865 12812 raft_consensus.cc:374] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.263878 12809 consensus_queue.cc:260] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.263911 12812 raft_consensus.cc:729] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2576c4ecf142aaa8a87da872d027c7, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.264020 12812 consensus_queue.cc:260] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.264122 12809 ts_tablet_manager.cc:1428] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:03.264122 12663 heartbeater.cc:503] Master 127.7.137.62:33117 was elected leader, sending a full tablet report...
I20260619 02:41:03.264292 12812 ts_tablet_manager.cc:1428] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:03.264322 12794 heartbeater.cc:503] Master 127.7.137.62:33117 was elected leader, sending a full tablet report...
I20260619 02:41:03.319509 12815 raft_consensus.cc:482] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:03.319660 12815 raft_consensus.cc:504] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.320043 12815 leader_election.cc:288] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a2576c4ecf142aaa8a87da872d027c7 (127.7.137.3:34827), 99d117cdef6e440bba8bf34541167c2c (127.7.137.1:33703)
I20260619 02:41:03.321534 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.320260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:34827 (local address 127.7.137.2:39579)
0619 02:41:03.320329 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.320343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.320384 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.320534 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.320536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.320538 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.321166 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.321167 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.321183 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.321193 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.321389 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.321390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.321390 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.321443 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.321456 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:41:03.321599 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.320339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:39579 (local address 127.7.137.3:34827)
0619 02:41:03.320402 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.320404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.320413 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:03.320443 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.320444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.320448 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.320527 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.321277 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.321279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.321279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.321283 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.321298 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.321303 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.321390 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.321477 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.321495 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:41:03.321784 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.320557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:48499 (local address 127.7.137.1:33703)
0619 02:41:03.320610 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.320612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.320640 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:41:03.320699 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.320700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.320704 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.320773 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.321472 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.321473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.321473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.321477 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.321492 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.321496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.321547 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.321694 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.321712 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:41:03.321897 12748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf9b793966d84df39c4784410519a1af" candidate_uuid: "bc3a23512b474d49892ce9def0de4167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2576c4ecf142aaa8a87da872d027c7" is_pre_election: true
I20260619 02:41:03.321964 12817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.320402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33703 (local address 127.7.137.2:48499)
0619 02:41:03.320585 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.320595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.320646 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.320800 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.320801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.320804 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:03.321351 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:03.321351 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:03.321358 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:03.321371 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:03.321580 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:03.321581 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:03.321581 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 02:41:03.321873 (+ 292us) client_negotiation.cc:240] Negotiation successful
0619 02:41:03.321888 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":110,"threads_started":1}
I20260619 02:41:03.322064 12748 raft_consensus.cc:2454] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc3a23512b474d49892ce9def0de4167 in term 0.
I20260619 02:41:03.322293 12552 leader_election.cc:302] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a2576c4ecf142aaa8a87da872d027c7, bc3a23512b474d49892ce9def0de4167; no voters:
I20260619 02:41:03.322299 12486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf9b793966d84df39c4784410519a1af" candidate_uuid: "bc3a23512b474d49892ce9def0de4167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d117cdef6e440bba8bf34541167c2c" is_pre_election: true
I20260619 02:41:03.322422 12486 raft_consensus.cc:2454] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc3a23512b474d49892ce9def0de4167 in term 0.
I20260619 02:41:03.322427 12815 raft_consensus.cc:2781] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:03.322466 12815 raft_consensus.cc:482] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:03.322496 12815 raft_consensus.cc:3037] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.323189 12815 raft_consensus.cc:504] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.323310 12815 leader_election.cc:288] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [CANDIDATE]: Term 1 election: Requested vote from peers 7a2576c4ecf142aaa8a87da872d027c7 (127.7.137.3:34827), 99d117cdef6e440bba8bf34541167c2c (127.7.137.1:33703)
I20260619 02:41:03.323496 12486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf9b793966d84df39c4784410519a1af" candidate_uuid: "bc3a23512b474d49892ce9def0de4167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d117cdef6e440bba8bf34541167c2c"
I20260619 02:41:03.323565 12486 raft_consensus.cc:3037] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.324177 12486 raft_consensus.cc:2454] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc3a23512b474d49892ce9def0de4167 in term 1.
I20260619 02:41:03.324278 12748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf9b793966d84df39c4784410519a1af" candidate_uuid: "bc3a23512b474d49892ce9def0de4167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a2576c4ecf142aaa8a87da872d027c7"
I20260619 02:41:03.324371 12748 raft_consensus.cc:3037] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.324390 12552 leader_election.cc:302] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99d117cdef6e440bba8bf34541167c2c, bc3a23512b474d49892ce9def0de4167; no voters:
I20260619 02:41:03.324509 12815 raft_consensus.cc:2781] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:03.324676 12815 raft_consensus.cc:686] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 1 LEADER]: Becoming Leader. State: Replica: bc3a23512b474d49892ce9def0de4167, State: Running, Role: LEADER
I20260619 02:41:03.324775 12815 consensus_queue.cc:237] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } }
I20260619 02:41:03.325399 12748 raft_consensus.cc:2454] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc3a23512b474d49892ce9def0de4167 in term 1.
I20260619 02:41:03.325729 12344 catalog_manager.cc:5495] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc3a23512b474d49892ce9def0de4167 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "bc3a23512b474d49892ce9def0de4167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc3a23512b474d49892ce9def0de4167" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:03.329311 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.328360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.7.137.2:34463)
0619 02:41:03.328398 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.328399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.328408 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:03.328492 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.328492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.328501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.328540 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.329164 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.329165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.329165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.329168 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.329180 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.329183 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.329226 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.329259 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.329269 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:03.330969 12486 raft_consensus.cc:1261] T cf9b793966d84df39c4784410519a1af P 99d117cdef6e440bba8bf34541167c2c [term 1 FOLLOWER]: Refusing update from remote peer bc3a23512b474d49892ce9def0de4167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:03.330979 12748 raft_consensus.cc:1261] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 1 FOLLOWER]: Refusing update from remote peer bc3a23512b474d49892ce9def0de4167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:03.331262 12818 consensus_queue.cc:1035] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d117cdef6e440bba8bf34541167c2c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:03.331382 12815 consensus_queue.cc:1035] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a2576c4ecf142aaa8a87da872d027c7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:03.333482 12821 mvcc.cc:205] Tried to move back new op lower bound from 7298403792201056256 to 7298403792178995200. Current Snapshot: MvccSnapshot[applied={T|T < 7298403792201056256}]
I20260619 02:41:03.333840 12822 mvcc.cc:205] Tried to move back new op lower bound from 7298403792201056256 to 7298403792178995200. Current Snapshot: MvccSnapshot[applied={T|T < 7298403792201056256}]
I20260619 02:41:03.334004 12823 mvcc.cc:205] Tried to move back new op lower bound from 7298403792201056256 to 7298403792178995200. Current Snapshot: MvccSnapshot[applied={T|T < 7298403792201056256}]
I20260619 02:41:03.336253 12343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:03.336328 12343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:03.337037 12343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d749e00d3254141888e11dfcb0d96e3: Sending DeleteTablet for 3 replicas of tablet cf9b793966d84df39c4784410519a1af
I20260619 02:41:03.337340 12597 tablet_service.cc:1514] Processing DeleteTablet for tablet cf9b793966d84df39c4784410519a1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:03 UTC) from {username='slave'} at 127.0.0.1:54974
I20260619 02:41:03.337415 12728 tablet_service.cc:1514] Processing DeleteTablet for tablet cf9b793966d84df39c4784410519a1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:03 UTC) from {username='slave'} at 127.0.0.1:42088
I20260619 02:41:03.337459 12466 tablet_service.cc:1514] Processing DeleteTablet for tablet cf9b793966d84df39c4784410519a1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:03 UTC) from {username='slave'} at 127.0.0.1:60340
I20260619 02:41:03.337589 12832 tablet_replica.cc:330] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: stopping tablet replica
I20260619 02:41:03.337683 12832 raft_consensus.cc:2229] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:03.337811 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12404
I20260619 02:41:03.339334 12831 tablet_replica.cc:330] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: stopping tablet replica
I20260619 02:41:03.339435 12831 raft_consensus.cc:2229] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:03.339563 12831 raft_consensus.cc:2258] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:03.339991 12831 ts_tablet_manager.cc:1905] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:03.340050 12832 raft_consensus.cc:2258] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:03.340497 12832 ts_tablet_manager.cc:1905] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:03.341951 12831 ts_tablet_manager.cc:1918] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:03.342025 12831 log.cc:1196] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/cf9b793966d84df39c4784410519a1af
I20260619 02:41:03.342254 12832 ts_tablet_manager.cc:1918] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:03.342319 12831 ts_tablet_manager.cc:1939] T cf9b793966d84df39c4784410519a1af P bc3a23512b474d49892ce9def0de4167: Deleting consensus metadata
I20260619 02:41:03.342419 12832 log.cc:1196] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/cf9b793966d84df39c4784410519a1af
I20260619 02:41:03.342687 12331 catalog_manager.cc:4841] TS bc3a23512b474d49892ce9def0de4167 (127.7.137.2:34463): tablet cf9b793966d84df39c4784410519a1af (table test-table [id=a9e368a289774f1b89cf77e57e81c708]) successfully deleted
I20260619 02:41:03.342743 12832 ts_tablet_manager.cc:1939] T cf9b793966d84df39c4784410519a1af P 7a2576c4ecf142aaa8a87da872d027c7: Deleting consensus metadata
I20260619 02:41:03.343227 12330 catalog_manager.cc:4841] TS 7a2576c4ecf142aaa8a87da872d027c7 (127.7.137.3:34827): tablet cf9b793966d84df39c4784410519a1af (table test-table [id=a9e368a289774f1b89cf77e57e81c708]) successfully deleted
W20260619 02:41:03.343823 12330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:33703 (error 108)
I20260619 02:41:03.344321 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.344195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33703 (local address 127.0.0.1:60342)
0619 02:41:03.344245 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.344289 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:41:03.344354 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12535
W20260619 02:41:03.344432 12330 catalog_manager.cc:4584] TS 99d117cdef6e440bba8bf34541167c2c (127.7.137.1:33703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf9b793966d84df39c4784410519a1af: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33703: connect: Connection refused (error 111)
I20260619 02:41:03.351429 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12666
I20260619 02:41:03.355711 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12313
2026-06-19T02:41:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (617 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 02:41:03.375346 7716 mini_oidc.cc:152] Starting JWKS server
I20260619 02:41:03.376395 7716 webserver.cc:460] Webserver started at https://127.0.0.1:33101/ using document root <none> and password file <none>
I20260619 02:41:03.376497 7716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:41:03.376675 7716 webserver.cc:460] Webserver started at http://127.0.0.1:40371/ using document root <none> and password file <none>
2026-06-19T02:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:03Z Disabled control of system clock
I20260619 02:41:03.384254 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37371
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37371
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33101/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/testchainca.cert with env {}
W20260619 02:41:03.462165 12840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:03.462366 12840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:03.462387 12840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:03.463441 12840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:03.463474 12840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:03.463487 12840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:03.463500 12840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:03.463511 12840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:41:03.463523 12840 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:41:03.463534 12840 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33101/jwks.json
I20260619 02:41:03.464988 12840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33101/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:03.465170 12840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:03.465329 12840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:03.470067 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.470072 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.470104 12840 server_base.cc:1034] running on GCE node
W20260619 02:41:03.470088 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.470557 12840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:03.470880 12840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:03.472041 12840 hybrid_clock.cc:648] HybridClock initialized: now 1781836863472024 us; error 48 us; skew 500 ppm
I20260619 02:41:03.473217 12840 webserver.cc:460] Webserver started at http://127.7.137.62:35737/ using document root <none> and password file <none>
I20260619 02:41:03.473449 12840 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:03.473517 12840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:03.473645 12840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:03.474627 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "b4422eb3ae99436d8ea299c6cf0646ef"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "518f32596c8016aeea6d258b1d397d34"
server_key_iv: "3f988a4e3b1055c2f785f76df3aedeaf"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.474982 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "b4422eb3ae99436d8ea299c6cf0646ef"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "518f32596c8016aeea6d258b1d397d34"
server_key_iv: "3f988a4e3b1055c2f785f76df3aedeaf"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.476338 12840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:03.477077 12854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.477242 12840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.477344 12840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "b4422eb3ae99436d8ea299c6cf0646ef"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "518f32596c8016aeea6d258b1d397d34"
server_key_iv: "3f988a4e3b1055c2f785f76df3aedeaf"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.477411 12840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.485778 12840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.486043 12840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.486184 12840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.490062 12840 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37371
I20260619 02:41:03.490092 12906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37371 every 8 connection(s)
I20260619 02:41:03.490446 12840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:03.491400 12907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.493656 12907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef: Bootstrap starting.
I20260619 02:41:03.494241 12907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.494529 12907 log.cc:824] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.495260 12907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef: No bootstrap required, opened a new log
I20260619 02:41:03.496873 12907 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:41:03.497036 12907 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.497082 12907 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4422eb3ae99436d8ea299c6cf0646ef, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.497242 12907 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:41:03.497320 12907 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:03.497360 12907 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:03.497411 12907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.497916 12907 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:41:03.498036 12907 leader_election.cc:302] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4422eb3ae99436d8ea299c6cf0646ef; no voters:
I20260619 02:41:03.498247 12907 leader_election.cc:288] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:03.498281 12910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:03.498394 12910 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [term 1 LEADER]: Becoming Leader. State: Replica: b4422eb3ae99436d8ea299c6cf0646ef, State: Running, Role: LEADER
I20260619 02:41:03.498387 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12840
I20260619 02:41:03.498502 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:03.498483 12910 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:41:03.498556 12907 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:03.498695 7716 external_mini_cluster.cc:1428] Setting key 7ba5187346aa3c84c0470fa13713571e
I20260619 02:41:03.498781 12910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4422eb3ae99436d8ea299c6cf0646ef. Latest consensus state: current_term: 1 leader_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } } }
I20260619 02:41:03.498847 12910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:03.499078 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4422eb3ae99436d8ea299c6cf0646ef" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } } }
I20260619 02:41:03.499229 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:03.499778 12914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:03.500326 12914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:03.502122 12914 catalog_manager.cc:1347] Generated new cluster ID: 22d580f9deda4647adcf22634c54d304
I20260619 02:41:03.502173 12914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:03.504920 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.499173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.7.137.62:37371)
0619 02:41:03.499357 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.499366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.499610 (+ 244us) server_negotiation.cc:407] Connection header received
0619 02:41:03.499816 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.499824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.499903 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.500063 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.500395 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.501658 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.502721 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.503339 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.503470 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.504164 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.504170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.504171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.504377 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.504418 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.504433 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.504506 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.504577 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.504657 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260619 02:41:03.509645 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:41:03.519345 12914 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:03.519980 12914 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:03.526780 12914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4422eb3ae99436d8ea299c6cf0646ef: Generated new TSK 0
I20260619 02:41:03.526971 12914 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:41:03.548519 12937 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 02:41:03.548636 12937 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 02:41:03.549149 12936 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:03.533539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37371 (local address 127.0.0.1:35356)
0619 02:41:03.533755 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.533767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.533838 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.534002 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.534003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.534004 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:41:03.534083 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.534086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.534705 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.534706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.535652 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.535655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.536018 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.536019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.536085 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.536108 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:41:03.548797 (+ 12689us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:41:03.548890 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.137.62:37371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260619 02:41:03.549464 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.533586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.7.137.62:37371)
0619 02:41:03.533657 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.533660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.533830 (+ 170us) server_negotiation.cc:407] Connection header received
0619 02:41:03.533927 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.533928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.533938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.533995 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:41:03.534159 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.534629 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.535754 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.535937 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.536000 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.536209 (+ 209us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:41:03.548625 (+ 12416us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:41:03.549373 (+ 748us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 02:41:03.549518 12916 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.137.62:37371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33101/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 02:41:03.549609 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12840
2026-06-19T02:41:03Z chronyd exiting
I20260619 02:41:03.585466 7716 test_util.cc:175] -----------------------------------------------
I20260619 02:41:03.585542 7716 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836839601721-7716-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 02:41:03.586243 7716 mini_oidc.cc:152] Starting JWKS server
I20260619 02:41:03.586871 7716 webserver.cc:460] Webserver started at https://127.0.0.1:44211/ using document root <none> and password file <none>
I20260619 02:41:03.586912 7716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:41:03.587059 7716 webserver.cc:460] Webserver started at http://127.0.0.1:43399/ using document root <none> and password file <none>
2026-06-19T02:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:03Z Disabled control of system clock
I20260619 02:41:03.594161 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35443
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:35443
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44211/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/testchainca.cert with env {}
W20260619 02:41:03.673396 12943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:03.673616 12943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:03.673655 12943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:03.674786 12943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:03.674851 12943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:03.674875 12943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:03.674904 12943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:03.674933 12943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:41:03.674962 12943 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:41:03.674989 12943 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44211/jwks.json
I20260619 02:41:03.676451 12943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:35443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44211/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:03.676751 12943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:03.676977 12943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:03.681514 12948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.681571 12949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.681654 12943 server_base.cc:1034] running on GCE node
W20260619 02:41:03.681695 12951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.681958 12943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:03.682237 12943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:03.683399 12943 hybrid_clock.cc:648] HybridClock initialized: now 1781836863683395 us; error 36 us; skew 500 ppm
I20260619 02:41:03.684620 12943 webserver.cc:460] Webserver started at http://127.7.137.62:43799/ using document root <none> and password file <none>
I20260619 02:41:03.684900 12943 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:03.684971 12943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:03.685120 12943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:03.686133 12943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "f97e4879a25b4536b6d68ace9c3273bd"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "5c3b92f7b9673736c63e73bbc1ea6449"
server_key_iv: "f6d0aa6a0d7b4265c35741195fe3ae78"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.686537 12943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "f97e4879a25b4536b6d68ace9c3273bd"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "5c3b92f7b9673736c63e73bbc1ea6449"
server_key_iv: "f6d0aa6a0d7b4265c35741195fe3ae78"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.687812 12943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:03.688649 12957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.688841 12943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:03.688937 12943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "f97e4879a25b4536b6d68ace9c3273bd"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "5c3b92f7b9673736c63e73bbc1ea6449"
server_key_iv: "f6d0aa6a0d7b4265c35741195fe3ae78"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.689023 12943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.700150 12943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.700510 12943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.700656 12943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.704682 12943 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:35443
I20260619 02:41:03.704756 13009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:35443 every 8 connection(s)
I20260619 02:41:03.705081 12943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:03.706029 13010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.708176 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 12943
I20260619 02:41:03.708274 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:03.708410 13010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd: Bootstrap starting.
I20260619 02:41:03.708459 7716 external_mini_cluster.cc:1428] Setting key 7611b8dd934d1d1cec145991ebc04e63
I20260619 02:41:03.709141 13010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.709447 13010 log.cc:824] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.710114 13010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd: No bootstrap required, opened a new log
I20260619 02:41:03.711824 13010 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } }
I20260619 02:41:03.711977 13010 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.712023 13010 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97e4879a25b4536b6d68ace9c3273bd, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.712152 13010 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } }
I20260619 02:41:03.712245 13010 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:03.712287 13010 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:03.712335 13010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.712993 13010 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } }
I20260619 02:41:03.713104 13010 leader_election.cc:302] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97e4879a25b4536b6d68ace9c3273bd; no voters:
I20260619 02:41:03.713284 13010 leader_election.cc:288] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:03.713310 13015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:03.713449 13015 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [term 1 LEADER]: Becoming Leader. State: Replica: f97e4879a25b4536b6d68ace9c3273bd, State: Running, Role: LEADER
I20260619 02:41:03.713544 13015 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } }
I20260619 02:41:03.713577 13010 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:03.713876 13015 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97e4879a25b4536b6d68ace9c3273bd. Latest consensus state: current_term: 1 leader_uuid: "f97e4879a25b4536b6d68ace9c3273bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } } }
I20260619 02:41:03.713882 13016 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97e4879a25b4536b6d68ace9c3273bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97e4879a25b4536b6d68ace9c3273bd" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35443 } } }
I20260619 02:41:03.713986 13015 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:03.714164 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.708959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.7.137.62:35443)
0619 02:41:03.709149 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.709163 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.709189 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:41:03.709706 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.709712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.709768 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.709907 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.710088 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.711013 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.711800 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.712345 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.712472 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.713173 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.713177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.713177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.713386 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.713431 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.713446 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.713768 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.713803 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.713883 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 02:41:03.713994 13016 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:03.714387 13022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:03.714846 13022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:03.715106 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:41:03.716707 13022 catalog_manager.cc:1347] Generated new cluster ID: 6fe004cd495b46a6b0da49a5fcd85c82
I20260619 02:41:03.716784 13022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:03.751500 13022 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:03.752074 13022 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:03.761432 13022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97e4879a25b4536b6d68ace9c3273bd: Generated new TSK 0
I20260619 02:41:03.761623 13022 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:41:03.790474 13040 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 02:41:03.790592 13040 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:41:03.790745 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.776223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.7.137.62:35443)
0619 02:41:03.776271 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.776273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.776392 (+ 119us) server_negotiation.cc:407] Connection header received
0619 02:41:03.776454 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.776455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.776468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.776535 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:41:03.776655 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.777386 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.778159 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.778452 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.778526 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.778673 (+ 147us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:41:03.790495 (+ 11822us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44211/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:41:03.790657 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44211/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 02:41:03.790814 13013 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44211/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 02:41:03.790987 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:03.776170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:35443 (local address 127.0.0.1:33264)
0619 02:41:03.776318 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.776334 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.776386 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.776542 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.776543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.776545 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:41:03.776594 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.776595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.777484 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.777485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.778074 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.778076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.778530 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.778531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.778580 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.778584 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:41:03.790751 (+ 12167us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44211/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:41:03.790892 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.137.62:35443: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44211/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":101,"threads_started":1}
I20260619 02:41:03.791399 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 12943
2026-06-19T02:41:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (242 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 02:41:03.829212 7716 mini_oidc.cc:152] Starting JWKS server
I20260619 02:41:03.829912 7716 webserver.cc:460] Webserver started at https://127.0.0.1:45435/ using document root <none> and password file <none>
I20260619 02:41:03.829981 7716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:41:03.830150 7716 webserver.cc:460] Webserver started at http://127.0.0.1:37733/ using document root <none> and password file <none>
2026-06-19T02:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:03Z Disabled control of system clock
I20260619 02:41:03.837826 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37113
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37113
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45435/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:41:03.917452 13046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:03.917656 13046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:03.917678 13046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:03.918848 13046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:03.918892 13046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:03.918905 13046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:03.918918 13046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:03.918975 13046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:41:03.918996 13046 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:41:03.919030 13046 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45435/jwks.json
I20260619 02:41:03.920468 13046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45435/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:03.920743 13046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:03.921049 13046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:03.925803 13052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.925772 13054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:03.925788 13051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:03.926098 13046 server_base.cc:1034] running on GCE node
I20260619 02:41:03.926335 13046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:03.926656 13046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:03.927821 13046 hybrid_clock.cc:648] HybridClock initialized: now 1781836863927816 us; error 42 us; skew 500 ppm
I20260619 02:41:03.929054 13046 webserver.cc:460] Webserver started at http://127.7.137.62:38149/ using document root <none> and password file <none>
I20260619 02:41:03.929286 13046 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:03.929351 13046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:03.929486 13046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:03.930521 13046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "a48f6e36ccfe47a0ae77af4629673a72"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "c5973586b1931b4e92f516700082f0e9"
server_key_iv: "be076c9b60d20d5fb5f1edb380faef2b"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.930881 13046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "a48f6e36ccfe47a0ae77af4629673a72"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "c5973586b1931b4e92f516700082f0e9"
server_key_iv: "be076c9b60d20d5fb5f1edb380faef2b"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.932219 13046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:03.932940 13060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:03.933137 13046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:03.933239 13046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "a48f6e36ccfe47a0ae77af4629673a72"
format_stamp: "Formatted at 2026-06-19 02:41:03 on dist-test-slave-pdnb"
server_key: "c5973586b1931b4e92f516700082f0e9"
server_key_iv: "be076c9b60d20d5fb5f1edb380faef2b"
server_key_version: "encryptionkey@0"
I20260619 02:41:03.933310 13046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:03.944422 13046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:03.944828 13046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:03.944998 13046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:03.949141 13046 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37113
I20260619 02:41:03.949168 13112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37113 every 8 connection(s)
I20260619 02:41:03.949528 13046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:03.950160 13113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:03.951864 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13046
I20260619 02:41:03.951993 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:03.952175 7716 external_mini_cluster.cc:1428] Setting key efbd1fac9bb93164b8df3c5a2aa8dac3
I20260619 02:41:03.953205 13113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: Bootstrap starting.
I20260619 02:41:03.953893 13113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:03.954205 13113 log.cc:824] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:03.955032 13113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: No bootstrap required, opened a new log
I20260619 02:41:03.957079 13113 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } }
I20260619 02:41:03.957204 13113 raft_consensus.cc:374] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:03.957226 13113 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48f6e36ccfe47a0ae77af4629673a72, State: Initialized, Role: FOLLOWER
I20260619 02:41:03.957368 13113 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } }
I20260619 02:41:03.957441 13113 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:03.957463 13113 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:03.957522 13113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:03.958082 13113 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } }
I20260619 02:41:03.958212 13113 leader_election.cc:302] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48f6e36ccfe47a0ae77af4629673a72; no voters:
I20260619 02:41:03.958261 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.952685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.7.137.62:37113)
0619 02:41:03.952895 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.952904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.952938 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:41:03.953459 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.953467 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.953553 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.953708 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:03.953884 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.955159 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.955999 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.956661 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.956849 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.957557 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:03.957565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:03.957566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:03.957762 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:03.957794 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:03.957804 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:03.957877 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:03.957950 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:41:03.958029 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260619 02:41:03.958390 13113 leader_election.cc:288] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:03.958417 13118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:03.958549 13118 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [term 1 LEADER]: Becoming Leader. State: Replica: a48f6e36ccfe47a0ae77af4629673a72, State: Running, Role: LEADER
I20260619 02:41:03.958626 13113 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:03.958633 13118 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } }
I20260619 02:41:03.959007 13118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48f6e36ccfe47a0ae77af4629673a72. Latest consensus state: current_term: 1 leader_uuid: "a48f6e36ccfe47a0ae77af4629673a72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } } }
I20260619 02:41:03.959081 13118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:03.959342 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:41:03.960333 13119 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a48f6e36ccfe47a0ae77af4629673a72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48f6e36ccfe47a0ae77af4629673a72" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37113 } } }
W20260619 02:41:03.960402 13134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 02:41:03.960434 13119 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72 [sys.catalog]: This master's current role is: LEADER
W20260619 02:41:03.960443 13134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:41:03.960518 13125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:03.960665 13125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:03.962102 13125 catalog_manager.cc:1347] Generated new cluster ID: 0ff2224d3dea4a78a4beef6779159d80
I20260619 02:41:03.962157 13125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:03.973578 13125 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:03.974242 13125 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:03.977406 13125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a48f6e36ccfe47a0ae77af4629673a72: Generated new TSK 0
I20260619 02:41:03.977564 13125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:03.986541 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:03.982849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.7.137.62:37113)
0619 02:41:03.982895 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:03.982897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:03.983003 (+ 106us) server_negotiation.cc:407] Connection header received
0619 02:41:03.983058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:03.983059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:03.983069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:03.983127 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:41:03.983260 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.983728 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.984935 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.985152 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.985207 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.985391 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:41:03.986309 (+ 918us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:41:03.986437 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 02:41:03.986598 13116 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 02:41:03.986608 13142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:03.982763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37113 (local address 127.0.0.1:33514)
0619 02:41:03.982925 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:03.982936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:03.982998 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:03.983141 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:03.983142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:03.983144 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:41:03.983202 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.983204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.983813 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.983814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.984843 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:03.984847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:03.985232 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:03.985233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:03.985293 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:03.985300 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:41:03.986413 (+ 1113us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:41:03.986511 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.137.62:37113: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260619 02:41:03.986907 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13046
E20260619 02:41:03.998838 13143 webserver.cc:533] Webserver: sslize failed:
E20260619 02:41:03.998916 13143 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T02:41:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (195 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T02:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:04Z Disabled control of system clock
I20260619 02:41:04.031807 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:43581
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:43581
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/testchainca.cert with env {}
W20260619 02:41:04.112023 13147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:04.112226 13147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:04.112248 13147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:04.113463 13147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:04.113505 13147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:04.113518 13147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:04.113531 13147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:04.113543 13147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:41:04.113556 13147 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:41:04.113605 13147 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 02:41:04.115034 13147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:43581
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:43581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:04.115281 13147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:04.115492 13147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:04.120167 13153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:04.120236 13152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:04.120220 13155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:04.120257 13147 server_base.cc:1034] running on GCE node
I20260619 02:41:04.120546 13147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:04.120877 13147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:04.122021 13147 hybrid_clock.cc:648] HybridClock initialized: now 1781836864122011 us; error 31 us; skew 500 ppm
I20260619 02:41:04.123265 13147 webserver.cc:460] Webserver started at http://127.7.137.62:33291/ using document root <none> and password file <none>
I20260619 02:41:04.123498 13147 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:04.123545 13147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:04.123692 13147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:04.124783 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "714c6cb98181437a978341555f48d37e"
format_stamp: "Formatted at 2026-06-19 02:41:04 on dist-test-slave-pdnb"
server_key: "e67955a0063bb8fae05c8b523b0c4cc4"
server_key_iv: "d6af587bbc1a1140d19bb465bf4c35c1"
server_key_version: "encryptionkey@0"
I20260619 02:41:04.125149 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "714c6cb98181437a978341555f48d37e"
format_stamp: "Formatted at 2026-06-19 02:41:04 on dist-test-slave-pdnb"
server_key: "e67955a0063bb8fae05c8b523b0c4cc4"
server_key_iv: "d6af587bbc1a1140d19bb465bf4c35c1"
server_key_version: "encryptionkey@0"
I20260619 02:41:04.126398 13147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:04.127185 13161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:04.127370 13147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:04.127460 13147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "714c6cb98181437a978341555f48d37e"
format_stamp: "Formatted at 2026-06-19 02:41:04 on dist-test-slave-pdnb"
server_key: "e67955a0063bb8fae05c8b523b0c4cc4"
server_key_iv: "d6af587bbc1a1140d19bb465bf4c35c1"
server_key_version: "encryptionkey@0"
I20260619 02:41:04.127529 13147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:04.138949 13147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:04.139226 13147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:04.139370 13147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:04.143196 13147 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:43581
I20260619 02:41:04.143225 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:43581 every 8 connection(s)
I20260619 02:41:04.143553 13147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:04.144423 13214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:04.145862 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13147
I20260619 02:41:04.145968 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:04.146162 7716 external_mini_cluster.cc:1428] Setting key cc537f8a2c1192d0ca76a178112666ee
I20260619 02:41:04.147168 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e: Bootstrap starting.
I20260619 02:41:04.147874 13214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:04.148159 13214 log.cc:824] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:04.148927 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e: No bootstrap required, opened a new log
I20260619 02:41:04.151219 13214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } }
I20260619 02:41:04.151386 13214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:04.151427 13214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714c6cb98181437a978341555f48d37e, State: Initialized, Role: FOLLOWER
I20260619 02:41:04.151592 13214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } }
I20260619 02:41:04.151674 13214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:04.151695 13214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:04.151746 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:04.146632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.7.137.62:43581)
0619 02:41:04.146813 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:04.146820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:04.146842 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:41:04.147258 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:04.147267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:04.147339 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:04.147452 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:04.147685 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.148822 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.149910 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.150306 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.150393 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:04.151065 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:04.151068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:04.151068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:04.151227 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:04.151257 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:04.151268 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:04.151317 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:04.151436 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:41:04.151519 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260619 02:41:04.151759 13214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:04.152551 13214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } }
I20260619 02:41:04.152659 13214 leader_election.cc:302] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 714c6cb98181437a978341555f48d37e; no voters:
I20260619 02:41:04.152856 13214 leader_election.cc:288] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:04.152878 13219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:04.152894 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:41:04.152982 13219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [term 1 LEADER]: Becoming Leader. State: Replica: 714c6cb98181437a978341555f48d37e, State: Running, Role: LEADER
I20260619 02:41:04.153072 13219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } }
I20260619 02:41:04.153131 13214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:04.153384 13219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 714c6cb98181437a978341555f48d37e. Latest consensus state: current_term: 1 leader_uuid: "714c6cb98181437a978341555f48d37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } } }
I20260619 02:41:04.153462 13219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:04.153395 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "714c6cb98181437a978341555f48d37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714c6cb98181437a978341555f48d37e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43581 } } }
I20260619 02:41:04.153617 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:04.154286 13223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:04.154726 13223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:04.156210 13223 catalog_manager.cc:1347] Generated new cluster ID: dd366994a13646d2af8a8b10b622b2de
I20260619 02:41:04.156251 13223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:04.194427 13223 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:04.195037 13223 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:04.199085 13223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 714c6cb98181437a978341555f48d37e: Generated new TSK 0
I20260619 02:41:04.199265 13223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:04.292359 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:04.207596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.7.137.62:43581)
0619 02:41:04.207657 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:04.207659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:04.207743 (+ 84us) server_negotiation.cc:407] Connection header received
0619 02:41:04.207817 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:04.207818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:04.207828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:04.207886 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:41:04.208072 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.208522 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.209611 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.209785 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.209831 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:04.210013 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:41:04.292146 (+ 82133us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:41:04.292299 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 02:41:04.292426 13217 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 02:41:04.292536 13243 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:04.207532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:43581 (local address 127.0.0.1:38174)
0619 02:41:04.207680 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:04.207690 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:04.207750 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:04.207912 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:04.207912 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:04.207914 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:41:04.207981 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:04.207983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.208605 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.208606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:04.209519 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:04.209523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:04.209864 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:04.209865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:04.209921 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:04.209927 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:41:04.292283 (+ 82356us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:41:04.292399 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.137.62:43581: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260619 02:41:04.292869 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13147
2026-06-19T02:41:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (286 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:04 dist-test-slave-pdnb krb5kdc[13255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:04 dist-test-slave-pdnb krb5kdc[13255](info): set up 2 sockets
Jun 19 02:41:04 dist-test-slave-pdnb krb5kdc[13255](info): commencing operation
krb5kdc: starting...
W20260619 02:41:05.488916 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:05 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:05.601547 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45267
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:05.682968 13268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:05.683162 13268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:05.683219 13268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:05.684365 13268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:05.684405 13268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:05.684420 13268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:05.684432 13268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:05.684445 13268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:05.685909 13268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:05.686095 13268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:05.686264 13268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:41:05.691300 13268 server_base.cc:1034] running on GCE node
W20260619 02:41:05.691282 13273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:05.691282 13276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:05.691273 13274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:05.691834 13268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:05.692111 13268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:05.693252 13268 hybrid_clock.cc:648] HybridClock initialized: now 1781836865693229 us; error 36 us; skew 500 ppm
Jun 19 02:41:05 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:05.695717 13268 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:05.696151 13268 webserver.cc:460] Webserver started at http://127.7.137.62:34033/ using document root <none> and password file <none>
I20260619 02:41:05.696393 13268 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:05.696439 13268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:05.696585 13268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:05.697711 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "dd45307139db40f18d45e563ac165d5a"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "593fac5ceb51a9d499d62e473c50b3d8"
server_key_iv: "83a0a3a097ef8ef21b9f42a0d1a380bd"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.698073 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "dd45307139db40f18d45e563ac165d5a"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "593fac5ceb51a9d499d62e473c50b3d8"
server_key_iv: "83a0a3a097ef8ef21b9f42a0d1a380bd"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.699379 13268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:05.700090 13283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:05.700278 13268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:05.700367 13268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "dd45307139db40f18d45e563ac165d5a"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "593fac5ceb51a9d499d62e473c50b3d8"
server_key_iv: "83a0a3a097ef8ef21b9f42a0d1a380bd"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.700415 13268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:05.712500 13268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:05.714794 13268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:05.714906 13268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:05.718803 13268 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45267
I20260619 02:41:05.718861 13335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45267 every 8 connection(s)
I20260619 02:41:05.719192 13268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:05.720151 13336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:05.722509 13336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a: Bootstrap starting.
I20260619 02:41:05.723081 13336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:05.723376 13336 log.cc:824] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:05.724088 13336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a: No bootstrap required, opened a new log
I20260619 02:41:05.725700 13336 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } }
I20260619 02:41:05.725879 13336 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:05.725936 13336 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd45307139db40f18d45e563ac165d5a, State: Initialized, Role: FOLLOWER
I20260619 02:41:05.726009 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13268
I20260619 02:41:05.726143 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:05.726133 13336 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } }
I20260619 02:41:05.726214 13336 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:05.726253 13336 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:05.726303 13336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:05.726377 7716 external_mini_cluster.cc:1428] Setting key 73158676c17b83feb3fc046d167a99f2
I20260619 02:41:05.726871 13336 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } }
I20260619 02:41:05.727025 13336 leader_election.cc:302] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd45307139db40f18d45e563ac165d5a; no voters:
I20260619 02:41:05.727192 13336 leader_election.cc:288] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:05.727432 13341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:05.727460 13336 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:05.727560 13341 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [term 1 LEADER]: Becoming Leader. State: Replica: dd45307139db40f18d45e563ac165d5a, State: Running, Role: LEADER
I20260619 02:41:05.727650 13341 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } }
I20260619 02:41:05.727953 13341 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd45307139db40f18d45e563ac165d5a. Latest consensus state: current_term: 1 leader_uuid: "dd45307139db40f18d45e563ac165d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } } }
I20260619 02:41:05.728032 13341 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:05.728314 13349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:05.728569 13342 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd45307139db40f18d45e563ac165d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd45307139db40f18d45e563ac165d5a" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45267 } } }
I20260619 02:41:05.728734 13342 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:05.728849 13349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:05.730464 13349 catalog_manager.cc:1347] Generated new cluster ID: 4029003fbcc94e27a3f067f04eab7c78
I20260619 02:41:05.730522 13349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:41:05 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:05.738021 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:05.726901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.7.137.62:45267)
0619 02:41:05.727092 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:05.727101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:05.727135 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:41:05.727269 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:05.727276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:05.727336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:05.727523 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:05.728277 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.729913 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.730915 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.731368 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.731489 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:05.733838 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:05.733842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:05.733842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:05.733853 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:05.736247 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:05.736672 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:05.736676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:05.736676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:05.736839 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:05.737105 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:05.737106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:05.737107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:05.737366 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:05.737571 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:05.737644 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:41:05.737792 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260619 02:41:05.748241 13349 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:05.748876 13349 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:05.753377 13349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd45307139db40f18d45e563ac165d5a: Generated new TSK 0
I20260619 02:41:05.753597 13349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:41:05.787045 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:05.876055 13362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:05.876298 13362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:05.876339 13362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:05.877590 13362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:05.877655 13362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:05.877698 13362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:05.879205 13362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:05.879463 13362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:05.879693 13362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:05.884719 13367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:05.884711 13370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:05.884711 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:05.884819 13362 server_base.cc:1034] running on GCE node
I20260619 02:41:05.885260 13362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:05.885568 13362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:05.886751 13362 hybrid_clock.cc:648] HybridClock initialized: now 1781836865886734 us; error 37 us; skew 500 ppm
Jun 19 02:41:05 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:05.889472 13362 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:41:05.889911 13362 webserver.cc:460] Webserver started at http://127.7.137.1:42253/ using document root <none> and password file <none>
I20260619 02:41:05.890152 13362 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:05.890199 13362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:05.890352 13362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:05.891389 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "c6b65c27b7e44359a02abf8f46d88ce0"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "c5be4172ae3a07b7d5c7d1ddd46da4cf"
server_key_iv: "82920baef7d96dae7fa698ccf219b34f"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.891738 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "c6b65c27b7e44359a02abf8f46d88ce0"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "c5be4172ae3a07b7d5c7d1ddd46da4cf"
server_key_iv: "82920baef7d96dae7fa698ccf219b34f"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.893136 13362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:05.893829 13377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:05.893986 13362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:05.894088 13362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "c6b65c27b7e44359a02abf8f46d88ce0"
format_stamp: "Formatted at 2026-06-19 02:41:05 on dist-test-slave-pdnb"
server_key: "c5be4172ae3a07b7d5c7d1ddd46da4cf"
server_key_iv: "82920baef7d96dae7fa698ccf219b34f"
server_key_version: "encryptionkey@0"
I20260619 02:41:05.894184 13362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:05.914839 13362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:05.916687 13362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:05.916864 13362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:05.917119 13362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:05.917424 13362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:05.917479 13362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:05.917519 13362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:05.917562 13362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:05.923338 13362 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45183
I20260619 02:41:05.923415 13490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45183 every 8 connection(s)
I20260619 02:41:05.923735 13362 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:41:05 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:05.932449 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13362
I20260619 02:41:05.932567 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:05.932823 7716 external_mini_cluster.cc:1428] Setting key ef946b5884102d9dffedfbf7fe478ee5
I20260619 02:41:05.933468 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:05.924510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:58233 (local address 127.7.137.62:45267)
0619 02:41:05.924572 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:05.924575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:05.924779 (+ 204us) server_negotiation.cc:407] Connection header received
0619 02:41:05.925714 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:05.925715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:05.925725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:05.925784 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:05.926403 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.926982 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.928129 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.928318 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.928372 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:05.930483 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:05.930484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:05.930484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:05.930493 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:05.931907 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:05.932335 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:05.932336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:05.932337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:05.932397 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:05.932746 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:05.932747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:05.932747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:05.932870 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:05.932950 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:05.933290 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 02:41:05.933379 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:41:05.933671 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:05.924470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45267 (local address 127.7.137.1:58233)
0619 02:41:05.924639 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:05.924682 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:05.925625 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:05.925837 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:05.925841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:05.926137 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:05.926329 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:05.926333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.927066 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.927067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:05.928058 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:05.928061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:05.928391 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:05.928391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:05.928495 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:05.928984 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:05.928992 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:05.930394 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:05.931992 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:05.931998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:05.932001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:05.932228 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:05.932472 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:05.932473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:05.932474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:05.932605 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:05.932977 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:05.932983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:05.933147 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 02:41:05.933263 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:41:05.933418 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 02:41:05.934094 13491 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45267
I20260619 02:41:05.934185 13491 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:05.934379 13491 heartbeater.cc:511] Master 127.7.137.62:45267 requested a full tablet report, sending...
I20260619 02:41:05.934927 13300 ts_manager.cc:194] Registered new tserver with Master: c6b65c27b7e44359a02abf8f46d88ce0 (127.7.137.1:45183)
I20260619 02:41:05.935859 13300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:58233
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:41:05.955448 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:06.036476 13496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:06.036701 13496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:06.036749 13496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:06.037881 13496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:06.037914 13496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:06.037940 13496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:06.039376 13496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:06.039616 13496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:06.039826 13496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:06.044931 13501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:06.044919 13502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:06.044943 13496 server_base.cc:1034] running on GCE node
W20260619 02:41:06.044919 13504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:06.045320 13496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:06.045573 13496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:06.046737 13496 hybrid_clock.cc:648] HybridClock initialized: now 1781836866046726 us; error 45 us; skew 500 ppm
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.049254 13496 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:41:06.049738 13496 webserver.cc:460] Webserver started at http://127.7.137.2:43063/ using document root <none> and password file <none>
I20260619 02:41:06.049973 13496 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:06.050040 13496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:06.050203 13496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:06.051247 13496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "c08608f2ab574557af3cce926cc50d58"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "918cbb3c4b0bbf727ba5d72339081493"
server_key_iv: "e8e429fed0af8b0bfe226e96c3c2277e"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.051607 13496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "c08608f2ab574557af3cce926cc50d58"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "918cbb3c4b0bbf727ba5d72339081493"
server_key_iv: "e8e429fed0af8b0bfe226e96c3c2277e"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.052977 13496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:06.053769 13511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.053969 13496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:06.054066 13496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "c08608f2ab574557af3cce926cc50d58"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "918cbb3c4b0bbf727ba5d72339081493"
server_key_iv: "e8e429fed0af8b0bfe226e96c3c2277e"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.054157 13496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:06.066905 13496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:06.069082 13496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:06.069255 13496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:06.069497 13496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:06.069820 13496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:06.069875 13496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.069916 13496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:06.069947 13496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.075961 13496 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:42671
I20260619 02:41:06.076013 13624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:42671 every 8 connection(s)
I20260619 02:41:06.076383 13496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:06.080353 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13496
I20260619 02:41:06.080454 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:06.080623 7716 external_mini_cluster.cc:1428] Setting key bba6911661219558518ffd0913223eb9
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:06.087318 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.077306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:46761 (local address 127.7.137.62:45267)
0619 02:41:06.077401 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.077403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.077566 (+ 163us) server_negotiation.cc:407] Connection header received
0619 02:41:06.078586 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.078587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.078607 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.078660 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.079301 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.079810 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.081272 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.081568 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.081659 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.084118 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:06.084119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:06.084119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:06.084128 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:06.085936 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.086333 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.086334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.086335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.086403 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.086650 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.086651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.086652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.086801 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:06.086876 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:06.087123 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 02:41:06.087219 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:41:06.087357 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.077196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45267 (local address 127.7.137.2:46761)
0619 02:41:06.077431 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:06.077479 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:06.078459 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:06.078736 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:06.078740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:06.079001 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:06.079199 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:06.079203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.079906 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.079908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:06.081192 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:06.081195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.081668 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.081670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:06.081802 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.082359 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:06.082367 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:06.084015 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:06.086030 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:06.086035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:06.086037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:06.086259 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:06.086474 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:06.086475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:06.086475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:06.086583 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:06.086869 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:06.086873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:06.086987 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:41:06.087067 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:41:06.087194 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":104,"threads_started":1}
I20260619 02:41:06.087992 13625 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45267
I20260619 02:41:06.088085 13625 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:06.088287 13625 heartbeater.cc:511] Master 127.7.137.62:45267 requested a full tablet report, sending...
I20260619 02:41:06.088707 13300 ts_manager.cc:194] Registered new tserver with Master: c08608f2ab574557af3cce926cc50d58 (127.7.137.2:42671)
I20260619 02:41:06.089146 13300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:46761
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:41:06.107116 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:06.187887 13630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:06.188088 13630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:06.188112 13630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:06.189225 13630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:06.189265 13630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:06.189291 13630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:06.190689 13630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:06.190874 13630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:06.191052 13630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:06.195888 13635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:06.195861 13638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:06.195868 13636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:06.196372 13630 server_base.cc:1034] running on GCE node
I20260619 02:41:06.196652 13630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:06.196964 13630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:06.198105 13630 hybrid_clock.cc:648] HybridClock initialized: now 1781836866198092 us; error 50 us; skew 500 ppm
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.200688 13630 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:41:06.201157 13630 webserver.cc:460] Webserver started at http://127.7.137.3:38439/ using document root <none> and password file <none>
I20260619 02:41:06.201400 13630 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:06.201485 13630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:06.201642 13630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:06.202775 13630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "8cd2af85354d4e5b811e7ad385143724"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "9f26d13fab6043ada2cd88a64fe09b54"
server_key_iv: "ab155f2c90b985d69b1c3fd3985a6dbd"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.203128 13630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "8cd2af85354d4e5b811e7ad385143724"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "9f26d13fab6043ada2cd88a64fe09b54"
server_key_iv: "ab155f2c90b985d69b1c3fd3985a6dbd"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.204583 13630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:06.205507 13645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.205713 13630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:06.205818 13630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "8cd2af85354d4e5b811e7ad385143724"
format_stamp: "Formatted at 2026-06-19 02:41:06 on dist-test-slave-pdnb"
server_key: "9f26d13fab6043ada2cd88a64fe09b54"
server_key_iv: "ab155f2c90b985d69b1c3fd3985a6dbd"
server_key_version: "encryptionkey@0"
I20260619 02:41:06.205907 13630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:06.217326 13630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:06.219542 13630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:06.219697 13630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:06.219916 13630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:06.220201 13630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:06.220250 13630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.220290 13630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:06.220317 13630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:06.226152 13630 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45329
I20260619 02:41:06.226194 13758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45329 every 8 connection(s)
I20260619 02:41:06.226496 13630 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:06.232402 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13630
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:06.232479 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:06.232657 7716 external_mini_cluster.cc:1428] Setting key b50cfb15814a698788e7a28c65cab17e
I20260619 02:41:06.235715 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.227146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:38287 (local address 127.7.137.62:45267)
0619 02:41:06.227186 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.227188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.227377 (+ 189us) server_negotiation.cc:407] Connection header received
0619 02:41:06.228362 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.228363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.228376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.228440 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.229081 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.229570 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.230572 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.230753 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.230811 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.232994 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:06.232996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:06.232997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:06.233038 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:06.234388 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.234786 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.234787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.234788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.234847 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.235094 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.235095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.235095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.235206 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:06.235284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:06.235552 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:41:06.235628 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:06.235888 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.227104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45267 (local address 127.7.137.3:38287)
0619 02:41:06.227249 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:06.227289 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:06.228263 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:06.228516 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:06.228519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:06.228821 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:06.229005 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:06.229009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.229657 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.229658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:06.230488 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:06.230490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.230826 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.230827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:06.230915 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.231396 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:06.231404 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:06.232920 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:06.234486 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:06.234492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:06.234494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:06.234712 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:06.234918 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:06.234919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:06.234919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:06.235029 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:06.235275 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:06.235280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:06.235411 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 02:41:06.235523 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:41:06.235663 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260619 02:41:06.236342 13759 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45267
I20260619 02:41:06.236415 13759 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:06.236604 13759 heartbeater.cc:511] Master 127.7.137.62:45267 requested a full tablet report, sending...
I20260619 02:41:06.236986 13300 ts_manager.cc:194] Registered new tserver with Master: 8cd2af85354d4e5b811e7ad385143724 (127.7.137.3:45329)
I20260619 02:41:06.237317 13300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:38287
I20260619 02:41:06.244269 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/test-admin.keytab.
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.258536 7716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 02:41:06.260079 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:06.259387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44188 (local address 127.7.137.1:45183)
0619 02:41:06.259551 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.259556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.259566 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:06.259598 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.259600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.259613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.259676 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.259998 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44188: BlockingRecv error: recv got EOF from 127.0.0.1:44188 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
W20260619 02:41:06.260208 7716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.261225 7716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.262077 7716 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:41:06.269603 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.262525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.7.137.1:45183)
0619 02:41:06.262560 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.262566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.262609 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:41:06.262647 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.262647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.262656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.262705 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.263078 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.263816 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.264517 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.264891 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.264949 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.266968 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:06.266974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:06.266974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:06.266981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:06.268591 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.268971 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.268974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.268974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.269038 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.269192 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.269192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.269192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.269301 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:06.269396 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:06.269455 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:41:06.269530 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:41:06.271359 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:06.270764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.7.137.1:45183)
0619 02:41:06.270818 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.270820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.270873 (+ 53us) server_negotiation.cc:407] Connection header received
0619 02:41:06.270943 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.270944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.270953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.271023 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.271296 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44218: BlockingRecv error: recv got EOF from 127.0.0.1:44218 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 02:41:06.271482 7716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.272377 7716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.273239 7716 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:41:06.279873 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.273714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.7.137.1:45183)
0619 02:41:06.273767 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.273769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.273832 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:41:06.273892 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.273893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.273902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.273968 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.274321 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.274818 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.275499 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.275703 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.275776 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.277890 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:06.277891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:06.277891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:06.277896 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:06.279016 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.279324 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.279324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.279325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.279370 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.279534 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.279535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.279535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.279613 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:06.279692 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:06.279764 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 02:41:06.279818 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 02:41:06.281490 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:06.280856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.7.137.1:45183)
0619 02:41:06.280915 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.280917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.281030 (+ 113us) server_negotiation.cc:407] Connection header received
0619 02:41:06.281099 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.281100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.281109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.281175 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.281439 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44242: BlockingRecv error: recv got EOF from 127.0.0.1:44242 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 02:41:06.281597 7716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836839601721-7716-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.282449 7716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:06.283231 7716 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:41:06.289557 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:06.283665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.7.137.1:45183)
0619 02:41:06.283714 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:06.283720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:06.283745 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:41:06.283789 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:06.283789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:06.283796 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:06.283837 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:06.284132 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.284553 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.285225 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:06.285414 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:06.285479 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:06.287377 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:06.287378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:06.287378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:06.287384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:06.288618 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.288991 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.288992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.288992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.289067 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:06.289231 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:06.289231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:06.289232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:06.289309 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:06.289371 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:06.289452 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:41:06.289501 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:41:06.290185 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13362
I20260619 02:41:06.294582 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13496
I20260619 02:41:06.298636 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13630
I20260619 02:41:06.302699 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13268
2026-06-19T02:41:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1838 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13779](info): set up 2 sockets
Jun 19 02:41:06 dist-test-slave-pdnb krb5kdc[13779](info): commencing operation
krb5kdc: starting...
W20260619 02:41:07.348979 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab.
Entry for principal oryx/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab.
I20260619 02:41:07.460289 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:34665
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab
--principal=oryx/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:07.539880 13792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:07.540113 13792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:07.540149 13792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:07.541350 13792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:07.541415 13792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:07.541441 13792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:07.541461 13792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:07.541492 13792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:07.542927 13792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:34665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab
--principal=oryx/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:34665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:07.543205 13792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:07.543423 13792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:07.548197 13797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:07.548197 13798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:07.548231 13800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:07.548588 13792 server_base.cc:1034] running on GCE node
I20260619 02:41:07.548866 13792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:07.549109 13792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:07.550253 13792 hybrid_clock.cc:648] HybridClock initialized: now 1781836867550238 us; error 29 us; skew 500 ppm
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:07.556790 13792 init.cc:377] Logged in from keytab as oryx/127.7.137.62@KRBTEST.COM (short username oryx)
I20260619 02:41:07.557263 13792 webserver.cc:460] Webserver started at http://127.7.137.62:45175/ using document root <none> and password file <none>
I20260619 02:41:07.557451 13792 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:07.557497 13792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:07.557590 13792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:07.558633 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "08d4a8dac973410cbc95a25d55480d02"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "5cfa2d03ad5aa44e7939fe976d25061c"
server_key_iv: "25c04906ab5bfca83fff52408fc13fb5"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.558929 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "08d4a8dac973410cbc95a25d55480d02"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "5cfa2d03ad5aa44e7939fe976d25061c"
server_key_iv: "25c04906ab5bfca83fff52408fc13fb5"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.560158 13792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:07.560894 13807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.561081 13792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:07.561179 13792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "08d4a8dac973410cbc95a25d55480d02"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "5cfa2d03ad5aa44e7939fe976d25061c"
server_key_iv: "25c04906ab5bfca83fff52408fc13fb5"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.561259 13792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:07.585646 13792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:07.587420 13792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:07.587586 13792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:07.591601 13792 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:34665
I20260619 02:41:07.591663 13859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:34665 every 8 connection(s)
I20260619 02:41:07.592011 13792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:07.593016 13860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:07.594910 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13792
I20260619 02:41:07.595026 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:07.595228 7716 external_mini_cluster.cc:1428] Setting key 76d0072987708e645313d4bd470f2c36
I20260619 02:41:07.595544 13860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Bootstrap starting.
I20260619 02:41:07.596273 13860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:07.596561 13860 log.cc:824] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:07.597229 13860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: No bootstrap required, opened a new log
I20260619 02:41:07.598901 13860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } }
I20260619 02:41:07.599056 13860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:07.599100 13860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08d4a8dac973410cbc95a25d55480d02, State: Initialized, Role: FOLLOWER
I20260619 02:41:07.599256 13860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } }
I20260619 02:41:07.599345 13860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:07.599387 13860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:07.599434 13860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:07.599925 13860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } }
I20260619 02:41:07.600040 13860 leader_election.cc:302] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08d4a8dac973410cbc95a25d55480d02; no voters:
I20260619 02:41:07.600239 13860 leader_election.cc:288] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:07.600260 13865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:07.600366 13865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 LEADER]: Becoming Leader. State: Replica: 08d4a8dac973410cbc95a25d55480d02, State: Running, Role: LEADER
I20260619 02:41:07.600443 13865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } }
I20260619 02:41:07.600489 13860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:07.600790 13865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } }
I20260619 02:41:07.600864 13865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:07.601016 13865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08d4a8dac973410cbc95a25d55480d02. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } }
I20260619 02:41:07.601091 13865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:07.601438 13870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:07.601918 13870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:07.603427 13870 catalog_manager.cc:1347] Generated new cluster ID: 65ec451cfc7543abbae75321a7541b09
I20260619 02:41:07.603478 13870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:07.605703 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:07.595699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.7.137.62:34665)
0619 02:41:07.595878 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:07.595888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:07.595923 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:41:07.596378 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:07.596393 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:07.596447 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:07.596591 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:07.597233 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.598153 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.598883 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.599416 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.599514 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:07.602414 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:07.602419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:07.602420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:07.602433 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:07.604178 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.604544 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.604547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.604548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.604635 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.604874 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.604874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.604875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.605106 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:07.605251 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:07.605358 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 02:41:07.605489 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260619 02:41:07.614576 13870 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:07.615113 13870 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:07.618602 13870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Generated new TSK 0
I20260619 02:41:07.618767 13870 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
I20260619 02:41:07.645444 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:07.741314 13886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:07.741511 13886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:07.741533 13886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:07.742808 13886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:07.742852 13886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:07.742879 13886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:07.744340 13886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:07.744575 13886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:07.744863 13886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:07.751026 13894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:07.751055 13892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:07.751165 13891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:07.751072 13886 server_base.cc:1034] running on GCE node
I20260619 02:41:07.751442 13886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:07.751693 13886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:07.752847 13886 hybrid_clock.cc:648] HybridClock initialized: now 1781836867752828 us; error 36 us; skew 500 ppm
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:07.755424 13886 init.cc:377] Logged in from keytab as oryx/127.7.137.1@KRBTEST.COM (short username oryx)
I20260619 02:41:07.755889 13886 webserver.cc:460] Webserver started at http://127.7.137.1:43479/ using document root <none> and password file <none>
I20260619 02:41:07.756137 13886 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:07.756208 13886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:07.756356 13886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:07.757468 13886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "b14589babd74421ca6286633352959fc"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "07ceab1981fc3e120af0dafd3318045e"
server_key_iv: "455281df51e4da512061aabc020a417b"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.757864 13886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "b14589babd74421ca6286633352959fc"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "07ceab1981fc3e120af0dafd3318045e"
server_key_iv: "455281df51e4da512061aabc020a417b"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.759245 13886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:07.759990 13901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.760166 13886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:07.760263 13886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "b14589babd74421ca6286633352959fc"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "07ceab1981fc3e120af0dafd3318045e"
server_key_iv: "455281df51e4da512061aabc020a417b"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.760345 13886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:07.775138 13886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:07.777293 13886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:07.777458 13886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:07.777693 13886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:07.778005 13886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:07.778062 13886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.778105 13886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:07.778132 13886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.784134 13886 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45719
I20260619 02:41:07.784211 14014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45719 every 8 connection(s)
I20260619 02:41:07.784533 13886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:07.790475 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 13886
I20260619 02:41:07.790616 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:07.790812 7716 external_mini_cluster.cc:1428] Setting key 2de48133abd6143820daf0d719322e74
I20260619 02:41:07.793896 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:07.785340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:56817 (local address 127.7.137.62:34665)
0619 02:41:07.785391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:07.785394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:07.785663 (+ 269us) server_negotiation.cc:407] Connection header received
0619 02:41:07.786108 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:07.786109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:07.786119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:07.786171 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:07.787076 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.787585 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.788567 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.788794 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.788862 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:07.790967 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:07.790969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:07.790969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:07.790976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:07.792512 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.793045 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.793046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.793046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.793111 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.793334 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.793335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.793336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.793469 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:07.793539 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:07.793771 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 02:41:07.793832 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:07.794060 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:07.785325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:56817)
0619 02:41:07.785542 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:07.785578 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:07.786028 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:07.786224 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:07.786227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:07.786799 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:07.786984 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:07.786990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.787664 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.787665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:07.788454 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:07.788456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.788871 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.788872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:07.788974 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:07.789389 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:07.789396 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:07.790841 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:07.792626 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:07.792634 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:07.792636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:07.792933 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:07.793181 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:07.793181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:07.793181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:07.793273 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:07.793533 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:07.793537 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:07.793645 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:41:07.793725 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:41:07.793838 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":109,"threads_started":1}
I20260619 02:41:07.794517 14015 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:07.794605 14015 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:07.794790 14015 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:07.795274 13824 ts_manager.cc:194] Registered new tserver with Master: b14589babd74421ca6286633352959fc (127.7.137.1:45719)
I20260619 02:41:07.795866 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:56817
WARNING: no policy specified for oryx/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
I20260619 02:41:07.815517 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:07.896279 14020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:07.896483 14020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:07.896507 14020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:07.897581 14020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:07.897617 14020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:07.897643 14020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:07.898977 14020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:07.899155 14020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:07.899327 14020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:07.904970 14028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:07.904982 14026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:07.905054 14020 server_base.cc:1034] running on GCE node
W20260619 02:41:07.904981 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:07.905463 14020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:07.905669 14020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:07.906803 14020 hybrid_clock.cc:648] HybridClock initialized: now 1781836867906796 us; error 31 us; skew 500 ppm
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:07.909305 14020 init.cc:377] Logged in from keytab as oryx/127.7.137.2@KRBTEST.COM (short username oryx)
I20260619 02:41:07.909803 14020 webserver.cc:460] Webserver started at http://127.7.137.2:33191/ using document root <none> and password file <none>
I20260619 02:41:07.909991 14020 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:07.910030 14020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:07.910127 14020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:07.911193 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "cf36019923a9410f8df8b695d1faec65"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "7a16633f40f54e6b02890be07cbdf47f"
server_key_iv: "7920bc5f630f0dbc269050857fb71a6d"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.911494 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "cf36019923a9410f8df8b695d1faec65"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "7a16633f40f54e6b02890be07cbdf47f"
server_key_iv: "7920bc5f630f0dbc269050857fb71a6d"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.912998 14020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:41:07.913750 14035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.913969 14020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:07.914075 14020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "cf36019923a9410f8df8b695d1faec65"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "7a16633f40f54e6b02890be07cbdf47f"
server_key_iv: "7920bc5f630f0dbc269050857fb71a6d"
server_key_version: "encryptionkey@0"
I20260619 02:41:07.914155 14020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:07.934604 14020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:07.937106 14020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:07.937276 14020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:07.937537 14020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:07.937849 14020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:07.937882 14020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.937927 14020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:07.937955 14020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:07.943462 14020 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:41261
I20260619 02:41:07.943531 14148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:41261 every 8 connection(s)
I20260619 02:41:07.943862 14020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:41:07 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:07.950227 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14020
I20260619 02:41:07.950342 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:07.950559 7716 external_mini_cluster.cc:1428] Setting key 503c49156adf644128a321ca5697de55
I20260619 02:41:07.953666 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:07.944646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:53995 (local address 127.7.137.62:34665)
0619 02:41:07.944716 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:07.944760 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:07.944908 (+ 148us) server_negotiation.cc:407] Connection header received
0619 02:41:07.945381 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:07.945382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:07.945392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:07.945453 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:07.946382 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.946887 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.947857 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.948070 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.948135 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:07.950337 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:07.950339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:07.950339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:07.950348 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:07.952201 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.952616 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.952618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.952618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.952682 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:07.953020 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:07.953021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:07.953021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:07.953135 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:07.953188 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:07.953532 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 02:41:07.953598 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:41:07.953976 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:07.944581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:53995)
0619 02:41:07.944774 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:07.944816 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:07.945284 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:07.945510 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:07.945513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:07.946081 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:07.946278 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:07.946283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.946977 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.946978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:07.947762 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:07.947763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:07.948150 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:07.948151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:07.948247 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:07.948754 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:07.948764 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:07.950192 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:07.952287 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:07.952294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:07.952296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:07.952543 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:07.952779 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:07.952781 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:07.952781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:07.952946 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:07.953201 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:07.953206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:07.953384 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 02:41:07.953514 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 02:41:07.953697 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260619 02:41:07.954516 14149 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:07.954608 14149 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:07.954810 14149 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:07.955209 13824 ts_manager.cc:194] Registered new tserver with Master: cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:07.955602 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:53995
WARNING: no policy specified for oryx/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
I20260619 02:41:07.975947 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:08.056604 14154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.056855 14154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.056883 14154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.058033 14154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.058076 14154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.058102 14154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:08.059490 14154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:34665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:08.059716 14154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.059937 14154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.064980 14162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.065018 14160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.065071 14154 server_base.cc:1034] running on GCE node
W20260619 02:41:08.065039 14159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.065382 14154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.065639 14154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.066795 14154 hybrid_clock.cc:648] HybridClock initialized: now 1781836868066792 us; error 36 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.069234 14154 init.cc:377] Logged in from keytab as oryx/127.7.137.3@KRBTEST.COM (short username oryx)
I20260619 02:41:08.069660 14154 webserver.cc:460] Webserver started at http://127.7.137.3:46053/ using document root <none> and password file <none>
I20260619 02:41:08.069875 14154 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.069941 14154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.070093 14154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:08.071185 14154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "608d55b69efe446691a329e44d7cf455"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "366ed1240f6d716907320bc98a6ccaaf"
server_key_iv: "b62fc4601f7393d6c8a47b1bc989282e"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.071550 14154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "608d55b69efe446691a329e44d7cf455"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "366ed1240f6d716907320bc98a6ccaaf"
server_key_iv: "b62fc4601f7393d6c8a47b1bc989282e"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.073009 14154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:08.073719 14169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.073887 14154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:08.073985 14154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "608d55b69efe446691a329e44d7cf455"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "366ed1240f6d716907320bc98a6ccaaf"
server_key_iv: "b62fc4601f7393d6c8a47b1bc989282e"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.074067 14154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.108901 14154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.110803 14154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:08.110984 14154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.111171 14154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:08.111474 14154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:08.111506 14154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.111529 14154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:08.111574 14154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.117957 14154 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:37593
I20260619 02:41:08.117987 14282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:37593 every 8 connection(s)
I20260619 02:41:08.118389 14154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:08.121169 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14154
I20260619 02:41:08.121235 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:08.121402 7716 external_mini_cluster.cc:1428] Setting key 1c44fb0e25475b432d1821e3a046e085
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:08.128295 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.119541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:42683 (local address 127.7.137.62:34665)
0619 02:41:08.119603 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.119605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.119820 (+ 215us) server_negotiation.cc:407] Connection header received
0619 02:41:08.120379 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.120380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.120393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.120443 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.121540 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.122288 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.123298 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.123511 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.123593 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.125685 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.125686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.125687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.125694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.126925 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.127324 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.127325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.127326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.127400 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.127662 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.127663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.127663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.127773 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.127843 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.128119 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.128200 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:41:08.128496 14285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.119547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:42683)
0619 02:41:08.119699 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.119735 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.120264 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.120514 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.120518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.121182 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.121430 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.121436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.122378 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.122379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.123196 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.123198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.123601 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.123602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.123733 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.124235 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.124243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.125589 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.127019 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.127024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.127027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.127248 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.127480 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.127480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.127481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.127582 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.127844 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.127851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.127985 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.128087 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.128263 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260619 02:41:08.128937 14283 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:08.129015 14283 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.129209 14283 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:08.129560 13824 ts_manager.cc:194] Registered new tserver with Master: 608d55b69efe446691a329e44d7cf455 (127.7.137.3:37593)
I20260619 02:41:08.129959 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:42683
I20260619 02:41:08.133212 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.137.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.61.keytab.
Entry for principal oryx/127.7.137.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.61.keytab.
I20260619 02:41:08.157439 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:33765
--webserver_interface=127.7.137.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.61.keytab
--principal=oryx/127.7.137.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:34665,127.7.137.61:33765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:08.239323 14288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.239522 14288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.239544 14288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.240694 14288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:08.240757 14288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.240772 14288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.240784 14288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:08.240796 14288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:08.242302 14288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:34665,127.7.137.61:33765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.61.keytab
--principal=oryx/127.7.137.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:33765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.137.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
W20260619 02:41:08.242425 14288 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.137.62:34665,127.7.137.61:33765'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 02:41:08.242581 14288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.242810 14288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:41:08.247653 14288 server_base.cc:1034] running on GCE node
W20260619 02:41:08.247617 14294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.247751 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.247627 14293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.248035 14288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.248265 14288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.249500 14288 hybrid_clock.cc:648] HybridClock initialized: now 1781836868249478 us; error 37 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.251914 14288 init.cc:377] Logged in from keytab as oryx/127.7.137.61@KRBTEST.COM (short username oryx)
I20260619 02:41:08.252318 14288 webserver.cc:460] Webserver started at http://127.7.137.61:33781/ using document root <none> and password file <none>
I20260619 02:41:08.252544 14288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.252588 14288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.252768 14288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:08.253806 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/instance:
uuid: "e15336f0c7684ca9a202817ec355dea5"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "598305d1a98e704ef1f55bd55f843a95"
server_key_iv: "a2bd8e172ecc2191029b61d205658404"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.254153 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal/instance:
uuid: "e15336f0c7684ca9a202817ec355dea5"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "598305d1a98e704ef1f55bd55f843a95"
server_key_iv: "a2bd8e172ecc2191029b61d205658404"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.255450 14288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:08.256343 14303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.256559 14288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:08.256628 14288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "e15336f0c7684ca9a202817ec355dea5"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "598305d1a98e704ef1f55bd55f843a95"
server_key_iv: "a2bd8e172ecc2191029b61d205658404"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.256754 14288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.265369 14288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.267058 14288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:08.267201 14288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.270853 14288 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.61:33765
I20260619 02:41:08.270882 14355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.61:33765 every 8 connection(s)
I20260619 02:41:08.271221 14288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
I20260619 02:41:08.271786 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14288
I20260619 02:41:08.272277 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:08.274581 14356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.61@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:08.282186 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.272400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.7.137.61:33765)
0619 02:41:08.272568 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.272575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.272600 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:41:08.273090 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.273096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.273148 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.273257 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.274026 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.274941 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.275881 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.276293 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.276404 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.278758 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.278765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.278765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.278771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.280664 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281074 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.281077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.281077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.281164 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281363 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.281364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.281364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.281597 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.281760 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.281851 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.281973 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 02:41:08.282253 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.275127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40514)
0619 02:41:08.275292 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.275328 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.275395 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.275539 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.275541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.275774 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.275837 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.275838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.276522 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.276523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.277320 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.277322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.277716 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.277717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.277788 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.278273 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.278281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.279616 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.281150 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.281158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.281342 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.281541 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.281543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.281599 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.281893 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.281899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.281944 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.282033 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.282150 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":119,"threads_started":1}
I20260619 02:41:08.282720 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.275204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.7.137.62:34665)
0619 02:41:08.275271 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.275273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.275399 (+ 126us) server_negotiation.cc:407] Connection header received
0619 02:41:08.275446 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.275447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.275458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.275517 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.275928 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.276419 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.277429 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.277627 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.277699 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.279707 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.279708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.279709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.279716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.281059 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281417 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.281418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.281418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.281475 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.281673 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.281674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.281674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.281816 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.281889 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.282066 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.282144 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:41:08.283190 14356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.61@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
I20260619 02:41:08.289237 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.283417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45322)
0619 02:41:08.283462 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.283473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.283507 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.283605 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.283605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.283709 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.283749 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.283750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.284361 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.284362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.284884 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.284886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.285219 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.285219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.285267 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.285584 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.285588 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.286708 (+ 1120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.288241 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.288242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.288242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.288397 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.288594 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.288595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.288595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.288646 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.288912 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.288913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.288947 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.289037 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.289128 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:41:08.289283 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.283462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.7.137.61:33765)
0619 02:41:08.283521 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.283523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.283529 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:41:08.283545 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.283546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.283552 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.283600 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.283833 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.284283 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.284951 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.285136 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.285221 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.286804 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.286805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.286805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.286811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.288147 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.288482 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.288483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.288483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.288532 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.288716 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.288744 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.288745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.288838 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.288909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.289058 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.289175 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":45}
I20260619 02:41:08.290448 14356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Bootstrap starting.
I20260619 02:41:08.291257 14356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:08.291529 14356 log.cc:824] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:08.292245 14356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: No bootstrap required, opened a new log
I20260619 02:41:08.293656 14356 raft_consensus.cc:348] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } }
I20260619 02:41:08.293778 14356 raft_consensus.cc:374] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:08.293825 14356 raft_consensus.cc:729] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15336f0c7684ca9a202817ec355dea5, State: Initialized, Role: FOLLOWER
I20260619 02:41:08.293951 14356 consensus_queue.cc:260] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } }
I20260619 02:41:08.294238 14356 sys_catalog.cc:564] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:08.294200 14362 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } } }
I20260619 02:41:08.294317 14362 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:41:08.295764 14373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:41:08.295851 14373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:41:08.297461 14288 master_runner.cc:428] Detected that this master e15336f0c7684ca9a202817ec355dea5 is joining an existing cluster
I20260619 02:41:08.297521 14288 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.137.61:33765
I20260619 02:41:08.305162 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.298694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.7.137.61:33765)
0619 02:41:08.298735 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.298736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.298842 (+ 106us) server_negotiation.cc:407] Connection header received
0619 02:41:08.298908 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.298909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.298918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.298970 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.299285 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.299919 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301066 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.301342 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301430 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.302484 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.302485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.302486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.302493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.304016 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304368 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.304369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.304370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.304417 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304617 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.304618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.304618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.304761 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.304823 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.304970 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.305048 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:41:08.305234 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.298663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45328)
0619 02:41:08.298768 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.298777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.298853 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.298985 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.298986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.299148 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.299212 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.299213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.299987 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.299988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.300949 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.300952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.301444 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.301502 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.301969 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.301974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.302416 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.304116 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.304118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.304285 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.304496 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.304497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.304553 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.304814 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.304815 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.304865 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.304951 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.305097 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":66,"thread_start_us":67,"threads_started":1}
I20260619 02:41:08.305522 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.298487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.7.137.62:34665)
0619 02:41:08.298552 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.298554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.298669 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:41:08.298791 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.298792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.298803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.298832 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.299560 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.300182 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301349 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.301647 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301718 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.302603 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.302604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.302604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.302611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.304295 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304700 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.304701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.304701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.304777 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304984 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.304984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.304985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.305105 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.305146 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.305346 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.305419 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:41:08.305821 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.298415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40528)
0619 02:41:08.298607 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.298617 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.298691 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.298892 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.298892 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.299399 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.299463 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.299465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.300258 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.300259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.301236 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.301238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.301726 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.301727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.301790 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.302154 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.302158 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.302520 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.304386 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.304388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.304606 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.304852 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.304853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.304853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.304913 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.305191 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.305192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.305228 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.305631 (+ 403us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.305725 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":98,"threads_started":1}
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.62@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
I20260619 02:41:08.314568 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.308001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.7.137.61:33765)
0619 02:41:08.308040 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.308042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.308214 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:41:08.308302 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.308303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.308312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.308367 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.308814 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.309255 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.310012 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.310204 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.310273 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.312117 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.312119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.312119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.312126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.313423 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.313803 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.313804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.313805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.313856 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.314051 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.314052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.314052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.314152 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.314221 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.314401 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.314473 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:08.314589 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.307952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45340)
0619 02:41:08.308124 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.308156 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.308239 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.308372 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.308375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.308645 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.308689 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.308690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.309333 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.309334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.309918 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.309920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.310272 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.310273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.310348 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.310777 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.310781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.312029 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.313504 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.313511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.313513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.313716 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.313922 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.313923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.313924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.313981 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.314225 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.314232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.314281 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.314376 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.314482 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260619 02:41:08.315140 13824 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.137.61:33765: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e15336f0c7684ca9a202817ec355dea5"
member_type: NON_VOTER
last_known_addr {
host: "127.7.137.61"
port: 33765
}
attrs {
promote: true
}
}
dest_uuid: "08d4a8dac973410cbc95a25d55480d02"
cas_config_opid_index: -1
I20260619 02:41:08.315442 13824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } }
I20260619 02:41:08.315904 14383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e15336f0c7684ca9a202817ec355dea5 (127.7.137.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } }
I20260619 02:41:08.316179 14330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:08.316915 14383 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.137.61:33765
I20260619 02:41:08.317088 14330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 02:41:08.317103 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } }
I20260619 02:41:08.317217 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:08.317420 14288 master.cc:537] Master@127.7.137.61:33765 shutting down...
I20260619 02:41:08.317415 14382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:41:08.319144 13811 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 02:41:08.319694 13811 consensus_peers.cc:487] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 -> Peer e15336f0c7684ca9a202817ec355dea5 (127.7.137.61:33765): Couldn't send request to peer e15336f0c7684ca9a202817ec355dea5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:41:08.320330 14288 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:08.320415 14288 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:08.320457 14288 tablet_replica.cc:330] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: stopping tablet replica
I20260619 02:41:08.332867 14288 master.cc:552] Master@127.7.137.61:33765 shutdown complete.
I20260619 02:41:08.332932 14288 master_runner.cc:305] Clearing existing system tablet
I20260619 02:41:08.333069 14288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.333127 14288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.333848 14288 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:08.334340 14387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.334497 14288 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.334561 14288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.334586 14288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "e15336f0c7684ca9a202817ec355dea5"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "598305d1a98e704ef1f55bd55f843a95"
server_key_iv: "a2bd8e172ecc2191029b61d205658404"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.335366 14288 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:08.337122 14288 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:41:08.337190 14288 log.cc:1196] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 02:41:08.337399 14288 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Deleting consensus metadata
I20260619 02:41:08.337538 14288 master_runner.cc:315] Copying system tablet from 127.7.137.62:34665
I20260619 02:41:08.338135 14288 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: tablet copy: Beginning tablet copy session from remote peer at address 127.7.137.62:34665
I20260619 02:41:08.344830 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.338521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.7.137.62:34665)
0619 02:41:08.338569 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.338571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.338705 (+ 134us) server_negotiation.cc:407] Connection header received
0619 02:41:08.338765 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.338766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.338775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.338821 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.339168 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.339658 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.340706 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.340999 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.341051 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.342209 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.342211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.342211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.342218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.343684 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.344057 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.344057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.344057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.344105 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.344306 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.344307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.344307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.344402 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.344451 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.344646 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.344701 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:41:08.344858 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.338425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40544)
0619 02:41:08.338622 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.338633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.338700 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.338838 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.338839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.339034 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.339102 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.339104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.339732 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.339734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.340622 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.340624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.341068 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.341069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.341138 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.341707 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.341712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.342087 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.343762 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.343763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.343764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.343983 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.344182 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.344183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.344183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.344246 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.344487 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.344487 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.344539 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.344624 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.344748 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260619 02:41:08.345479 13844 tablet_copy_service.cc:140] P 08d4a8dac973410cbc95a25d55480d02: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e15336f0c7684ca9a202817ec355dea5 ({username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:40544)
I20260619 02:41:08.345562 13844 tablet_copy_service.cc:161] P 08d4a8dac973410cbc95a25d55480d02: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e15336f0c7684ca9a202817ec355dea5 at {username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:40544: session id = e15336f0c7684ca9a202817ec355dea5-00000000000000000000000000000000
I20260619 02:41:08.346196 13844 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:41:08.346812 14288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:41:08.347712 14288 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: tablet copy: Starting download of 0 data blocks...
I20260619 02:41:08.347859 14288 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: tablet copy: Starting download of 1 WAL segments...
I20260619 02:41:08.348687 14288 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:41:08.349449 13844 tablet_copy_service.cc:342] P 08d4a8dac973410cbc95a25d55480d02: Request end of tablet copy session e15336f0c7684ca9a202817ec355dea5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:40544
I20260619 02:41:08.349505 13844 tablet_copy_service.cc:433] P 08d4a8dac973410cbc95a25d55480d02: ending tablet copy session e15336f0c7684ca9a202817ec355dea5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e15336f0c7684ca9a202817ec355dea5
I20260619 02:41:08.351751 14288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.352950 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.353056 14288 server_base.cc:1034] running on GCE node
W20260619 02:41:08.353170 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.353209 14397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.353364 14288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.353499 14288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.354633 14288 hybrid_clock.cc:648] HybridClock initialized: now 1781836868354623 us; error 28 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.355933 14288 init.cc:377] Logged in from keytab as oryx/127.7.137.61@KRBTEST.COM (short username oryx)
I20260619 02:41:08.356220 14288 webserver.cc:460] Webserver started at http://127.7.137.61:40763/ using document root <none> and password file <none>
I20260619 02:41:08.356346 14288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.356392 14288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.357087 14288 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:08.357621 14404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.357774 14288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:08.357839 14288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "e15336f0c7684ca9a202817ec355dea5"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "598305d1a98e704ef1f55bd55f843a95"
server_key_iv: "a2bd8e172ecc2191029b61d205658404"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.357899 14288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.364459 14288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.366168 14288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.369185 14288 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.61:33765
I20260619 02:41:08.369490 14456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.61:33765 every 8 connection(s)
I20260619 02:41:08.369606 14288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
I20260619 02:41:08.370432 14457 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:41:08.370919 14457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Bootstrap starting.
I20260619 02:41:08.373615 14457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:41:08.373845 14457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Bootstrap complete.
I20260619 02:41:08.374050 14457 raft_consensus.cc:348] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } }
I20260619 02:41:08.374138 14457 raft_consensus.cc:729] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e15336f0c7684ca9a202817ec355dea5, State: Initialized, Role: LEARNER
I20260619 02:41:08.374174 14457 consensus_queue.cc:260] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } }
I20260619 02:41:08.374326 14457 sys_catalog.cc:564] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:08.374336 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } }
I20260619 02:41:08.374480 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:41:08.375833 14471 catalog_manager.cc:1259] Loaded cluster ID: 65ec451cfc7543abbae75321a7541b09
I20260619 02:41:08.375880 14471 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: loading cluster ID for follower catalog manager: success
I20260619 02:41:08.376945 14471 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: acquiring CA information for follower catalog manager: success
I20260619 02:41:08.377262 14471 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:41:08.390731 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.383916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.7.137.61:33765)
0619 02:41:08.384026 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.384028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.384044 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:41:08.384066 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.384066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.384074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.384154 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.384862 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.385491 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.386501 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.386692 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.386740 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.388146 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.388147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.388147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.388153 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.389625 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.390005 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.390005 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.390006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.390054 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.390256 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.390256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.390257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.390365 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.390437 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.390598 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.390657 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.7.137.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.60.keytab.
Entry for principal oryx/127.7.137.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.60.keytab.
I20260619 02:41:08.413661 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:39015
--webserver_interface=127.7.137.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.60.keytab
--principal=oryx/127.7.137.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:08.494813 14475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.495012 14475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.495034 14475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.496111 14475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:08.496153 14475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.496168 14475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.496182 14475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:08.496194 14475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:08.497707 14475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.60.keytab
--principal=oryx/127.7.137.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:39015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.137.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:08.497895 14475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.498054 14475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.502941 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.502920 14480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.502905 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.503189 14475 server_base.cc:1034] running on GCE node
I20260619 02:41:08.503386 14475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.503618 14475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.504782 14475 hybrid_clock.cc:648] HybridClock initialized: now 1781836868504772 us; error 31 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.507220 14475 init.cc:377] Logged in from keytab as oryx/127.7.137.60@KRBTEST.COM (short username oryx)
I20260619 02:41:08.507691 14475 webserver.cc:460] Webserver started at http://127.7.137.60:38869/ using document root <none> and password file <none>
I20260619 02:41:08.507915 14475 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.507999 14475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.508150 14475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:08.509230 14475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/instance:
uuid: "59fbc2207cb14cb5b357213c604474bb"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "fc423582421fb72aedd97a1ce1a65544"
server_key_iv: "5ba78ea2a079da25209b98e2254fd566"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.509574 14475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal/instance:
uuid: "59fbc2207cb14cb5b357213c604474bb"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "fc423582421fb72aedd97a1ce1a65544"
server_key_iv: "5ba78ea2a079da25209b98e2254fd566"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.510866 14475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:08.511561 14490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.511723 14475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:08.511816 14475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "59fbc2207cb14cb5b357213c604474bb"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "fc423582421fb72aedd97a1ce1a65544"
server_key_iv: "5ba78ea2a079da25209b98e2254fd566"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.511894 14475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.520921 14475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.523092 14475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:08.523242 14475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.527272 14475 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.60:39015
I20260619 02:41:08.527310 14542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.60:39015 every 8 connection(s)
I20260619 02:41:08.527658 14475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
I20260619 02:41:08.528015 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14475
I20260619 02:41:08.528833 14543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:08.530831 14543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.60@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:08.538884 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.531601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.7.137.62:34665)
0619 02:41:08.531657 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.531659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.531771 (+ 112us) server_negotiation.cc:407] Connection header received
0619 02:41:08.531810 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.531811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.531823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.531875 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.532284 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.532861 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.533600 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.533958 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.534040 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.536111 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.536112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.536112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.536119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.537604 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.537978 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.537980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.537980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.538034 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.538254 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.538255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.538255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.538366 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.538436 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.538683 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.538761 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:41:08.539062 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.528646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.7.137.60:39015)
0619 02:41:08.528844 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.528853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.528888 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:41:08.529448 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.529457 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.529517 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.529673 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.530218 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.531466 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.532338 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.532763 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.532847 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.535609 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.535622 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.535623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.535629 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.537598 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.537952 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.537955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.537955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.538035 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.538256 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.538257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.538257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.538451 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.538629 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.538672 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.538787 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":122,"threads_started":1}
I20260619 02:41:08.540001 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.531456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40554)
0619 02:41:08.531675 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.531708 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.531763 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.531895 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.531896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.532142 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.532204 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.532205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.532953 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.532953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.533517 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.533519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.534047 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.534048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.534098 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.534576 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.534586 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.536024 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.537706 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.537710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.537712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.537884 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.538101 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.538102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.538103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.538189 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.538439 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.538445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.538523 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.538645 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.538872 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":86,"thread_start_us":103,"threads_started":1}
I20260619 02:41:08.540432 14543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.60@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
I20260619 02:41:08.546296 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.540670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45368)
0619 02:41:08.540739 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.540750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.540811 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.540953 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.540954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.541079 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.541124 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.541126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.541758 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.541758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.542244 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.542245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.542629 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.542630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.542697 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.543097 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.543102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.544189 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.545592 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.545593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.545593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.545744 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.545885 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.545885 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.545885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.545924 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.546088 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.546089 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.546130 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.546179 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.546241 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:41:08.546307 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.540782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.7.137.61:33765)
0619 02:41:08.540840 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.540842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.540850 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:08.540873 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.540873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.540885 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.540948 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.541200 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.541671 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.542323 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.542540 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.542624 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.544263 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.544264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.544265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.544271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.545524 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.545802 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.545802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.545803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.545846 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.545956 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.545957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.545957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.546039 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.546081 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.546194 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.546252 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:41:08.546648 14543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.60@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
I20260619 02:41:08.552665 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.546881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:54968)
0619 02:41:08.546923 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.546933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.546968 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.547070 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.547071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.547204 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.547225 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.547226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.547831 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.547832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.548279 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.548280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.548655 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.548656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.548703 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.549110 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.549121 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.550153 (+ 1032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.551700 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.551700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.551701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.551857 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.552052 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.552053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.552053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.552104 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.552346 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.552347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.552381 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.552448 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.552539 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:41:08.552711 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.546943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54968 (local address 127.7.137.60:39015)
0619 02:41:08.546975 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.546977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.546992 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:41:08.547009 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.547010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.547017 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.547067 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.547302 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.547744 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.548368 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.548571 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.548655 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.550249 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.550250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.550250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.550256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.551592 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.551938 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.551939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.551940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.551991 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.552180 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.552181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.552181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.552283 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.552354 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.552476 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.552586 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":18}
I20260619 02:41:08.554220 14543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Bootstrap starting.
I20260619 02:41:08.555392 14543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:08.555604 14543 log.cc:824] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:08.556273 14543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: No bootstrap required, opened a new log
I20260619 02:41:08.557756 14543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } }
I20260619 02:41:08.557889 14543 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:08.557914 14543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fbc2207cb14cb5b357213c604474bb, State: Initialized, Role: FOLLOWER
I20260619 02:41:08.558034 14543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } }
I20260619 02:41:08.558288 14543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:08.558274 14548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } } }
I20260619 02:41:08.558382 14548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:41:08.559896 14559 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:41:08.559953 14559 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:41:08.562163 14475 master_runner.cc:428] Detected that this master 59fbc2207cb14cb5b357213c604474bb is joining an existing cluster
I20260619 02:41:08.562234 14475 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.137.60:39015
I20260619 02:41:08.571122 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.7.137.61:33765)
0619 02:41:08.563888 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.563889 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.564165 (+ 276us) server_negotiation.cc:407] Connection header received
0619 02:41:08.564257 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.564258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.564268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.564297 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.564843 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565608 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566623 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.566916 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566956 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.568324 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.568325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.568326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.568331 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.569743 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.570124 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.570125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.570125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.570172 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.570682 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.570683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.570683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.570777 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.570826 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.570984 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.571047 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:08.571182 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45378)
0619 02:41:08.564108 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.564117 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.564189 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.564358 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.564358 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.564637 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.564751 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.564753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565705 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.565705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.566527 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.566530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.567032 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.567033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.567096 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.567478 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.567482 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.568238 (+ 756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.569822 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.569823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.569823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.570040 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.570552 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.570553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.570553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.570620 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.570835 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.570836 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.570880 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.570963 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.571068 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260619 02:41:08.588191 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.7.137.60:39015)
0619 02:41:08.563580 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.563582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.563743 (+ 161us) server_negotiation.cc:407] Connection header received
0619 02:41:08.563812 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.563812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.563821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.563874 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.564262 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565013 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566034 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.566303 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566336 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.585210 (+ 18874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.585211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.585212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.585220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.586824 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.587229 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.587230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.587231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.587306 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.587529 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.587530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.587530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.587671 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.587739 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.587943 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.588064 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":18}
I20260619 02:41:08.588387 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:54974)
0619 02:41:08.563689 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.563700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.563758 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.563888 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.563889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.564109 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.564185 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.564187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565107 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.565108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.565954 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.565956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.566409 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.566473 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.566944 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.566949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.585106 (+ 18157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.586927 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.586928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.586928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.587143 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.587389 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.587390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.587390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.587457 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.587770 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.587771 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.587820 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.587908 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.588277 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260619 02:41:08.589087 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.7.137.62:34665)
0619 02:41:08.563559 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.563561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.564144 (+ 583us) server_negotiation.cc:407] Connection header received
0619 02:41:08.564247 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.564248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.564260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.564290 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.564691 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565459 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.566502 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.567146 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.567191 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.585999 (+ 18808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.586001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.586001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.586017 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.587657 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.588025 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.588026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.588026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.588086 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.588356 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.588357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.588357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.588504 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.588547 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.588912 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.588995 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:41:08.589097 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.563629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40560)
0619 02:41:08.564074 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.564084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.564172 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.564348 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.564348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.564558 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.564605 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.564607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.565560 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.565560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.566417 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.566418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.567272 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.567272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.567341 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.585438 (+ 18097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.585441 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.585890 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.587731 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.587731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.587732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.587931 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.588159 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.588160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.588160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.588222 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.588711 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.588712 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.588767 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.588922 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.589030 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":78,"threads_started":1}
I20260619 02:41:08.590389 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13886
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836867, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.62@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
I20260619 02:41:08.598289 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:45719
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:34665
--webserver_port=43479
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:41:08.598482 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.591476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:54986)
0619 02:41:08.591553 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.591564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.591600 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.591833 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.591834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.591985 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.592029 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.592031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.592910 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.592911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.593691 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.593693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.594174 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.594176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.594237 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.594668 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.594673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.595851 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.597465 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.597466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.597467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.597636 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.597846 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.597847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.597847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.597907 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.598158 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.598159 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.598201 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.598290 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.598391 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 02:41:08.598479 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.591671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.7.137.60:39015)
0619 02:41:08.591704 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.591706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.591712 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:41:08.591741 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.591742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.591750 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.591792 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.592108 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.592823 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.593782 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.594068 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.594124 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.595937 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.595938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.595938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.595945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.597364 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.597720 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.597721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.597722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.597775 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.597978 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.597979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.597979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.598085 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.598166 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.598325 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.598390 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:08.599085 13824 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.137.60:39015: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "59fbc2207cb14cb5b357213c604474bb"
member_type: NON_VOTER
last_known_addr {
host: "127.7.137.60"
port: 39015
}
attrs {
promote: true
}
}
dest_uuid: "08d4a8dac973410cbc95a25d55480d02"
cas_config_opid_index: 5
I20260619 02:41:08.599318 13824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } }
I20260619 02:41:08.599653 14383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 59fbc2207cb14cb5b357213c604474bb (127.7.137.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.600558 14383 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.137.60:39015
I20260619 02:41:08.600642 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.600755 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:08.600844 14475 master.cc:537] Master@127.7.137.60:39015 shutting down...
I20260619 02:41:08.601099 14517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:08.602121 14517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:41:08.602391 14382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:41:08.603101 13808 consensus_peers.cc:487] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 -> Peer 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015): Couldn't send request to peer 59fbc2207cb14cb5b357213c604474bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:41:08.604147 14475 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:08.604234 14475 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:08.604277 14475 tablet_replica.cc:330] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: stopping tablet replica
I20260619 02:41:08.605098 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.599719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45384)
0619 02:41:08.600983 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.600994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.601045 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.601331 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.601332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.601334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:08.601373 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.601374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.603076 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.603077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.604086 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.604089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.604845 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.604846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.604917 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.604938 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.604998 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.605034 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248}
I20260619 02:41:08.605333 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.599812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45384 (local address 127.7.137.61:33765)
0619 02:41:08.599900 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.599902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.601103 (+ 1201us) server_negotiation.cc:407] Connection header received
0619 02:41:08.601134 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.601135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.601160 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.601221 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:08.601455 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.602166 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.604225 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.604771 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.604847 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.604921 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.605234 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.605271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:41:08.605537 14431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 LEARNER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:41:08.605711 14382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 02:41:08.606285 14431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 59fbc2207cb14cb5b357213c604474bb (127.7.137.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.607566 14475 master.cc:552] Master@127.7.137.60:39015 shutdown complete.
I20260619 02:41:08.607604 14475 master_runner.cc:305] Clearing existing system tablet
I20260619 02:41:08.607554 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08d4a8dac973410cbc95a25d55480d02. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.607650 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:41:08.607695 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: NON_VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: true } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.607733 14475 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.607746 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:41:08.607772 14475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.607820 14382 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: attempting to promote NON_VOTER e15336f0c7684ca9a202817ec355dea5 to VOTER
I20260619 02:41:08.607993 14382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } }
I20260619 02:41:08.608451 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.608361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:54994)
0619 02:41:08.608393 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.608426 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 02:41:08.608644 14431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 LEARNER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 02:41:08.608848 14475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:08.608911 14383 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 02:41:08.609324 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.609250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:54998)
0619 02:41:08.609282 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.609302 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 02:41:08.609457 13808 consensus_peers.cc:487] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 -> Peer 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015): Couldn't send request to peer 59fbc2207cb14cb5b357213c604474bb. Status: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:41:08.609473 14573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.609653 14475 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.609707 14475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:08.609735 14475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "59fbc2207cb14cb5b357213c604474bb"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "fc423582421fb72aedd97a1ce1a65544"
server_key_iv: "5ba78ea2a079da25209b98e2254fd566"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.609774 14383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e15336f0c7684ca9a202817ec355dea5 (127.7.137.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.610562 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.610625 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:08.610668 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.610709 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:08.610671 14475 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:08.610899 14431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e15336f0c7684ca9a202817ec355dea5 (127.7.137.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.611683 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.611748 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:41:08.612216 14475 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:41:08.612277 14475 log.cc:1196] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 02:41:08.612488 14475 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Deleting consensus metadata
I20260619 02:41:08.612607 14475 master_runner.cc:315] Copying system tablet from 127.7.137.62:34665
I20260619 02:41:08.613626 14475 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: tablet copy: Beginning tablet copy session from remote peer at address 127.7.137.62:34665
I20260619 02:41:08.620882 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.614025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.7.137.62:34665)
0619 02:41:08.614067 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.614069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.614138 (+ 69us) server_negotiation.cc:407] Connection header received
0619 02:41:08.614200 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.614200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.614212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.614288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.614626 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.615476 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.616534 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.616865 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.616943 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.618138 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.618139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.618140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.618147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.619722 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.620083 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.620084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.620084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.620143 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.620334 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.620335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.620335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.620450 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.620489 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.620672 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.620791 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:41:08.621459 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.613967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40566)
0619 02:41:08.614087 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.614096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.614150 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.614293 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.614294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.614499 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.614541 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.614543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.615567 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.615568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.616399 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.616401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.617013 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.617014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.617084 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.617603 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.617608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.617992 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.619808 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.619809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.619810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.620009 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.620210 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.620210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.620211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.620269 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.620529 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.620529 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.620571 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.621203 (+ 632us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.621352 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260619 02:41:08.621893 13844 tablet_copy_service.cc:140] P 08d4a8dac973410cbc95a25d55480d02: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 59fbc2207cb14cb5b357213c604474bb ({username='oryx', principal='oryx/127.7.137.60@KRBTEST.COM'} at 127.0.0.1:40566)
I20260619 02:41:08.621944 13844 tablet_copy_service.cc:161] P 08d4a8dac973410cbc95a25d55480d02: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 59fbc2207cb14cb5b357213c604474bb at {username='oryx', principal='oryx/127.7.137.60@KRBTEST.COM'} at 127.0.0.1:40566: session id = 59fbc2207cb14cb5b357213c604474bb-00000000000000000000000000000000
I20260619 02:41:08.622617 13844 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:41:08.623270 14475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:41:08.624442 14475 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: tablet copy: Starting download of 0 data blocks...
I20260619 02:41:08.624567 14475 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: tablet copy: Starting download of 1 WAL segments...
I20260619 02:41:08.625634 14475 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:41:08.626500 13844 tablet_copy_service.cc:342] P 08d4a8dac973410cbc95a25d55480d02: Request end of tablet copy session 59fbc2207cb14cb5b357213c604474bb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.137.60@KRBTEST.COM'} at 127.0.0.1:40566
I20260619 02:41:08.626547 13844 tablet_copy_service.cc:433] P 08d4a8dac973410cbc95a25d55480d02: ending tablet copy session 59fbc2207cb14cb5b357213c604474bb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 59fbc2207cb14cb5b357213c604474bb
I20260619 02:41:08.629102 14475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.630232 14580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.633773 14583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.633944 14475 server_base.cc:1034] running on GCE node
W20260619 02:41:08.637177 14581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.637329 14475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.637478 14475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.638631 14475 hybrid_clock.cc:648] HybridClock initialized: now 1781836868638619 us; error 30 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.640249 14475 init.cc:377] Logged in from keytab as oryx/127.7.137.60@KRBTEST.COM (short username oryx)
I20260619 02:41:08.640525 14475 webserver.cc:460] Webserver started at http://127.7.137.60:45441/ using document root <none> and password file <none>
I20260619 02:41:08.640645 14475 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.640679 14475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.641451 14475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:08.642035 14590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.642145 14475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:08.642182 14475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "59fbc2207cb14cb5b357213c604474bb"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "fc423582421fb72aedd97a1ce1a65544"
server_key_iv: "5ba78ea2a079da25209b98e2254fd566"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.642241 14475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.660390 14475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.662271 14475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.665397 14475 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.60:39015
I20260619 02:41:08.665766 14475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
I20260619 02:41:08.668864 14642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.60:39015 every 8 connection(s)
I20260619 02:41:08.669513 14643 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:41:08.670015 14643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Bootstrap starting.
I20260619 02:41:08.672922 14643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:41:08.673164 14643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: Bootstrap complete.
I20260619 02:41:08.673357 14643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } }
I20260619 02:41:08.673446 14643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59fbc2207cb14cb5b357213c604474bb, State: Initialized, Role: LEARNER
I20260619 02:41:08.673503 14643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } }
I20260619 02:41:08.673691 14643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:08.673684 14646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.673933 14646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: LEARNER
I20260619 02:41:08.675220 14657 catalog_manager.cc:1259] Loaded cluster ID: 65ec451cfc7543abbae75321a7541b09
I20260619 02:41:08.675310 14657 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: loading cluster ID for follower catalog manager: success
I20260619 02:41:08.676699 14657 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: acquiring CA information for follower catalog manager: success
I20260619 02:41:08.677067 14657 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:41:08.699061 14568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.699291 14568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.699331 14568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.700469 14568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.700539 14568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.700582 14568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:08.702332 14568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:45719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=43479
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:08.702597 14568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.702847 14568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.709010 14662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.709039 14663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.709039 14665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.710899 14568 server_base.cc:1034] running on GCE node
I20260619 02:41:08.711122 14568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.711400 14568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.712579 14568 hybrid_clock.cc:648] HybridClock initialized: now 1781836868712547 us; error 40 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.715405 14568 init.cc:377] Logged in from keytab as oryx/127.7.137.1@KRBTEST.COM (short username oryx)
I20260619 02:41:08.715883 14568 webserver.cc:460] Webserver started at http://127.7.137.1:43479/ using document root <none> and password file <none>
I20260619 02:41:08.716158 14568 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.716250 14568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.717825 14568 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:08.718513 14672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.718701 14568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:08.718803 14568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "b14589babd74421ca6286633352959fc"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "07ceab1981fc3e120af0dafd3318045e"
server_key_iv: "455281df51e4da512061aabc020a417b"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.719156 14568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.729055 14568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.730832 14568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:08.730993 14568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.731209 14568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:08.731496 14568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:08.731547 14568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.731586 14568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:08.731612 14568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.737741 14568 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45719
I20260619 02:41:08.737777 14785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45719 every 8 connection(s)
I20260619 02:41:08.738132 14568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:08.738564 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14568
I20260619 02:41:08.738737 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14020
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
I20260619 02:41:08.746272 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:41261
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:34665
--webserver_port=33191
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:41:08.751873 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.739086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53537 (local address 127.7.137.62:34665)
0619 02:41:08.739135 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.739138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.740072 (+ 934us) server_negotiation.cc:407] Connection header received
0619 02:41:08.740105 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.740106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.740127 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.740181 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.741071 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.741575 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.742987 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.743283 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.743325 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.748110 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.748111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.748111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.748122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.750091 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.750785 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.750786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.750787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.750848 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.751124 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.751125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.751125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.751283 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.751353 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.751691 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.751772 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:41:08.751986 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.738986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:55283 (local address 127.7.137.60:39015)
0619 02:41:08.739188 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.739190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.739418 (+ 228us) server_negotiation.cc:407] Connection header received
0619 02:41:08.739924 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.739925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.739937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.740042 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.741084 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.742235 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.743315 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.743607 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.743672 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.747171 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.747172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.747172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.747179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.749335 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.751151 (+ 1816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.751152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.751153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.751204 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.751392 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.751393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.751393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.751506 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.751579 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.751794 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.751869 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":46,"threads_started":1}
I20260619 02:41:08.752573 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.738884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:53537)
0619 02:41:08.739969 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.739978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.740027 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.740255 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.740258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.740829 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.740975 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.740979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.741704 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.741705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.742883 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.742885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.743405 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.743406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.743551 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.744306 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.744311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.746657 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.750538 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.750538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.750539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.750703 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.750928 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.750928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.750929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.751052 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.751361 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.751367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.751538 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.751652 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.751849 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":103,"threads_started":1}
I20260619 02:41:08.753057 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.738901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.7.137.1:55283)
0619 02:41:08.739261 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.739320 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.739820 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.740257 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.740258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.740700 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.740975 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.740981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.742315 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.742316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.743234 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.743236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.743724 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.743725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.743792 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.744201 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.744211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.747080 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.750173 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.750180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.750182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.750486 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.751272 (+ 786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.751273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.751273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.751327 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.751583 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.751584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.751627 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.751695 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.752901 (+ 1206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":1070,"thread_start_us":92,"threads_started":1}
I20260619 02:41:08.753083 14786 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:08.753183 14786 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.753389 14786 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:08.753844 14787 heartbeater.cc:347] Connected to a master server at 127.7.137.60:39015
I20260619 02:41:08.753904 14787 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.753880 13824 ts_manager.cc:194] Re-registered known tserver with Master: b14589babd74421ca6286633352959fc (127.7.137.1:45719)
I20260619 02:41:08.753978 14787 heartbeater.cc:511] Master 127.7.137.60:39015 requested a full tablet report, sending...
I20260619 02:41:08.754347 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:53537
I20260619 02:41:08.754662 14600 ts_manager.cc:194] Registered new tserver with Master: b14589babd74421ca6286633352959fc (127.7.137.1:45719)
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
I20260619 02:41:08.761904 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.746234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.7.137.1:42273)
0619 02:41:08.746788 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.746824 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.746938 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.747253 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.747254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.747504 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.747599 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.747601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.749384 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.749384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.750265 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.750267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.750999 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.751000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.751084 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.757057 (+ 5973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.757063 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.759097 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.760950 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.760951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.760952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.761143 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.761332 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.761333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.761333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.761381 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.761640 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.761640 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.761678 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.761730 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.761824 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":71,"threads_started":1}
I20260619 02:41:08.761945 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.746746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:42273 (local address 127.7.137.61:33765)
0619 02:41:08.746821 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.746823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.746909 (+ 86us) server_negotiation.cc:407] Connection header received
0619 02:41:08.747052 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.747053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.747097 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.747132 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.748484 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.749288 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.750399 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.750864 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.750952 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.759192 (+ 8240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.759194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.759195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.759203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.760845 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.761211 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.761212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.761212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.761271 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.761436 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.761437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.761437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.761582 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.761645 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.761770 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.761847 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:41:08.762530 14788 heartbeater.cc:347] Connected to a master server at 127.7.137.61:33765
I20260619 02:41:08.762589 14788 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.762684 14788 heartbeater.cc:511] Master 127.7.137.61:33765 requested a full tablet report, sending...
I20260619 02:41:08.763069 14421 ts_manager.cc:194] Registered new tserver with Master: b14589babd74421ca6286633352959fc (127.7.137.1:45719)
W20260619 02:41:08.833856 14794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.834052 14794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.834084 14794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.835251 14794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.835294 14794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.835321 14794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:08.836843 14794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:41261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=33191
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:08.837072 14794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.837313 14794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.842388 14799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.842388 14800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.842464 14802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.842548 14794 server_base.cc:1034] running on GCE node
I20260619 02:41:08.842801 14794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.843029 14794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.844195 14794 hybrid_clock.cc:648] HybridClock initialized: now 1781836868844175 us; error 45 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.846681 14794 init.cc:377] Logged in from keytab as oryx/127.7.137.2@KRBTEST.COM (short username oryx)
I20260619 02:41:08.847059 14794 webserver.cc:460] Webserver started at http://127.7.137.2:33191/ using document root <none> and password file <none>
I20260619 02:41:08.847229 14794 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.847277 14794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.848501 14794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:08.849084 14809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.849291 14794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.849395 14794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "cf36019923a9410f8df8b695d1faec65"
format_stamp: "Formatted at 2026-06-19 02:41:07 on dist-test-slave-pdnb"
server_key: "7a16633f40f54e6b02890be07cbdf47f"
server_key_iv: "7920bc5f630f0dbc269050857fb71a6d"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.849730 14794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.874432 14794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:08.876890 14794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:08.877045 14794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:08.877317 14794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:08.877643 14794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:08.877677 14794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.877699 14794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:08.877741 14794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.883208 14794 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:41261
I20260619 02:41:08.883292 14922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:41261 every 8 connection(s)
I20260619 02:41:08.883625 14794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:08.883932 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14794
I20260619 02:41:08.884102 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14154
I20260619 02:41:08.889633 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:37593
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:34665
--webserver_port=46053
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:08.899853 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.888912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:50885 (local address 127.7.137.60:39015)
0619 02:41:08.888958 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.888960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.888969 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:08.888998 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.888999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.889019 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.889077 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.890596 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891412 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.892758 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893091 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893158 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.896195 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.896196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.896196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.896205 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.898091 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.898628 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.898629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.898630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.898692 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.898996 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.898996 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.898997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.899145 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.899194 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.899606 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.899708 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:41:08.900529 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.889913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:39119 (local address 127.7.137.61:33765)
0619 02:41:08.889953 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.889955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.890405 (+ 450us) server_negotiation.cc:407] Connection header received
0619 02:41:08.890529 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.890531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.890544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.890578 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.890937 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891623 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.892913 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893207 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893263 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.897324 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.897327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.897328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.897335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.899106 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.899601 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.899602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.899602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.899655 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.899858 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.899859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.899859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.900022 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.900078 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.900370 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.900441 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:41:08.900537 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.884365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.7.137.2:50885)
0619 02:41:08.885079 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.885130 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.885933 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.889214 (+ 3281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.889219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.890119 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.890487 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.890493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891544 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.891545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.892627 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.892629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893222 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.893388 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.894124 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.894135 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.896096 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.898190 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.898197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.898200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.898531 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.898769 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.898770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.898771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.898911 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.899241 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.899247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.899420 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.899997 (+ 577us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.900215 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":107,"thread_start_us":71,"threads_started":1}
I20260619 02:41:08.900532 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.889761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.7.137.2:39119)
0619 02:41:08.890340 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.890351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.890431 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.890642 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.890643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.890795 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.890850 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.890851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891706 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.891707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.892818 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.892820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893314 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.893395 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.894149 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.894154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.897225 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.899207 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.899208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.899209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.899430 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.899725 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.899726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.899726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.899789 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.900118 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.900119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.900164 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.900228 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.900353 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":97,"thread_start_us":60,"threads_started":1}
I20260619 02:41:08.901302 14925 heartbeater.cc:347] Connected to a master server at 127.7.137.61:33765
I20260619 02:41:08.901404 14925 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.901566 14924 heartbeater.cc:347] Connected to a master server at 127.7.137.60:39015
I20260619 02:41:08.901623 14924 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.901698 14924 heartbeater.cc:511] Master 127.7.137.60:39015 requested a full tablet report, sending...
I20260619 02:41:08.901612 14925 heartbeater.cc:511] Master 127.7.137.61:33765 requested a full tablet report, sending...
I20260619 02:41:08.902207 14600 ts_manager.cc:194] Registered new tserver with Master: cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:08.902205 14421 ts_manager.cc:194] Registered new tserver with Master: cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:08.902199 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.884680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:45733 (local address 127.7.137.62:34665)
0619 02:41:08.884744 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.884746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.889430 (+ 4684us) server_negotiation.cc:407] Connection header received
0619 02:41:08.889564 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.889565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.889579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.889614 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:08.890605 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891416 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.892921 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893224 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893262 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.897866 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:08.897868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:08.897868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:08.897877 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:08.899725 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.901243 (+ 1518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.901244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.901244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.901308 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:08.901592 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:08.901593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:08.901594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:08.901727 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:08.901779 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.902036 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.902116 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:41:08.902621 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.884365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:45733)
0619 02:41:08.889354 (+ 4989us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.889367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.889463 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.890013 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.890014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.890175 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:08.890488 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.890496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.891538 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.891539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.892818 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.892820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.893332 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.893333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.893395 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.894262 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:08.894267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:08.897488 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:08.899806 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.899807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.899807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.901136 (+ 1329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.901416 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:08.901417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:08.901417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:08.901515 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:08.901848 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:08.901848 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:08.901906 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.902421 (+ 515us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.902519 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4962,"mutex_wait_us":724,"thread_start_us":65,"threads_started":1}
I20260619 02:41:08.903065 14923 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:08.903111 14923 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:08.903178 14923 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:08.903381 13824 ts_manager.cc:194] Re-registered known tserver with Master: cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:08.903890 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:45733
W20260619 02:41:08.974140 14930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:08.974375 14930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:08.974409 14930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:08.975816 14930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:08.975858 14930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:08.975885 14930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:08.977348 14930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:37593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=46053
--tserver_master_addrs=127.7.137.62:34665,127.7.137.61:33765,127.7.137.60:39015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:08.977584 14930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:08.977830 14930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:08.982821 14935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.982820 14938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:08.982815 14936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:08.983043 14930 server_base.cc:1034] running on GCE node
I20260619 02:41:08.983251 14930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:08.983434 14930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:08.984601 14930 hybrid_clock.cc:648] HybridClock initialized: now 1781836868984588 us; error 25 us; skew 500 ppm
Jun 19 02:41:08 dist-test-slave-pdnb krb5kdc[13779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:08.987160 14930 init.cc:377] Logged in from keytab as oryx/127.7.137.3@KRBTEST.COM (short username oryx)
I20260619 02:41:08.987639 14930 webserver.cc:460] Webserver started at http://127.7.137.3:46053/ using document root <none> and password file <none>
I20260619 02:41:08.987861 14930 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:08.987943 14930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:08.989428 14930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:08.990036 14945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:08.990234 14930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:08.990334 14930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "608d55b69efe446691a329e44d7cf455"
format_stamp: "Formatted at 2026-06-19 02:41:08 on dist-test-slave-pdnb"
server_key: "366ed1240f6d716907320bc98a6ccaaf"
server_key_iv: "b62fc4601f7393d6c8a47b1bc989282e"
server_key_version: "encryptionkey@0"
I20260619 02:41:08.990679 14930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:08.998379 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.995658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:55014)
0619 02:41:08.995820 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:08.995832 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:08.995913 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:08.996078 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:08.996078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:08.996080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:08.996136 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.996138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.996826 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.996827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.997578 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:08.997579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.998136 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.998137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:08.998197 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.998201 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:08.998259 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:41:08.998307 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260619 02:41:08.998378 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:08.995864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.7.137.60:39015)
0619 02:41:08.995929 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:08.995931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:08.995941 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:08.995971 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:08.995972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:08.996000 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:08.996073 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:08.996205 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.996690 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.997668 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:08.998054 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:08.998137 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:08.998206 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:08.998265 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:41:08.998305 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:41:08.998870 14617 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 02:41:08.999073 14646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08d4a8dac973410cbc95a25d55480d02. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:08.999214 14646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: LEARNER
I20260619 02:41:09.024811 14930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:09.026739 14930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:09.026846 14930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:09.027055 14930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:09.027379 14930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:09.027412 14930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:09.027436 14930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:09.027490 14930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:09.033264 14930 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:37593
I20260619 02:41:09.033334 15058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:37593 every 8 connection(s)
I20260619 02:41:09.033670 14930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:09.035950 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 14930
Jun 19 02:41:09 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.61@KRBTEST.COM
Jun 19 02:41:09 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
Jun 19 02:41:09 dist-test-slave-pdnb krb5kdc[13779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836868, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.60@KRBTEST.COM
I20260619 02:41:09.047360 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:38989)
0619 02:41:09.034743 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.034752 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.035759 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.036110 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.036115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.036599 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:09.036929 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.036951 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.038373 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.038373 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.039653 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.039655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040185 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.040318 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.041047 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:09.041052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:09.043737 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:09.045705 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.045710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.045712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.046011 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.046232 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.046232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.046233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.046352 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.046651 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:09.046657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:09.046817 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.046930 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.047096 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":77,"thread_start_us":52,"threads_started":1}
I20260619 02:41:09.047880 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:38989 (local address 127.7.137.62:34665)
0619 02:41:09.034728 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.034729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.034795 (+ 66us) server_negotiation.cc:407] Connection header received
0619 02:41:09.035871 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.035872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.035886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.035919 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.037049 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.037949 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.039723 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040052 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040120 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.043829 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.043831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.043831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.043838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.045623 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.046103 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.046103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.046104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.046161 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.046425 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.046426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.046426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.046564 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.046624 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.047710 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.047786 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:09.048288 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:33453 (local address 127.7.137.60:39015)
0619 02:41:09.034694 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.034696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.034805 (+ 109us) server_negotiation.cc:407] Connection header received
0619 02:41:09.035881 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.035883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.035901 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.035934 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.037031 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.037815 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040520 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040710 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040790 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.044301 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.044302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.044302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.044309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.047052 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047458 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.047459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.047460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.047524 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047727 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.047728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.047728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.047852 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.047893 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.048114 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.048196 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:41:09.048368 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:56299 (local address 127.7.137.61:33765)
0619 02:41:09.034639 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.034641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.034793 (+ 152us) server_negotiation.cc:407] Connection header received
0619 02:41:09.035871 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.035872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.035886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.035919 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.037169 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.037874 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.039919 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040206 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040242 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.044508 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.044509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.044509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.044516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.046492 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047602 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.047603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.047603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.047648 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047838 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.047839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.047839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.047957 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.047998 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.048187 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.048265 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260619 02:41:09.048434 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.7.137.3:56299)
0619 02:41:09.034727 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.034738 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.035769 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.036123 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.036123 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.037033 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:09.037085 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.037087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.038624 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.038625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.039837 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.039839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040308 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.040365 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.040941 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:09.040951 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:09.044385 (+ 3434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:09.047262 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.047263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.047517 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.047720 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047720 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.047721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.047770 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.048043 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:09.048044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:09.048081 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.048219 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.048310 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260619 02:41:09.048596 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.034343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.7.137.3:33453)
0619 02:41:09.034534 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.034567 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.035731 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.036112 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.036116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.036598 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:09.036930 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.036936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.038372 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.038373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.040429 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.040431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040875 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.040876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.040971 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.041327 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:09.041332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:09.044070 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:09.047139 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.047140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.047377 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.047599 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:09.047599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:09.047600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:09.047652 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:09.047936 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:09.047937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:09.047990 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.048394 (+ 404us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.048476 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":97,"threads_started":1}
I20260619 02:41:09.048843 15059 heartbeater.cc:347] Connected to a master server at 127.7.137.62:34665
I20260619 02:41:09.048929 15059 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:09.048985 15060 heartbeater.cc:347] Connected to a master server at 127.7.137.60:39015
I20260619 02:41:09.049037 15060 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:09.049111 15059 heartbeater.cc:511] Master 127.7.137.62:34665 requested a full tablet report, sending...
I20260619 02:41:09.049119 15060 heartbeater.cc:511] Master 127.7.137.60:39015 requested a full tablet report, sending...
I20260619 02:41:09.049140 15061 heartbeater.cc:347] Connected to a master server at 127.7.137.61:33765
I20260619 02:41:09.049368 15061 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:09.049432 15061 heartbeater.cc:511] Master 127.7.137.61:33765 requested a full tablet report, sending...
I20260619 02:41:09.049618 14600 ts_manager.cc:194] Registered new tserver with Master: 608d55b69efe446691a329e44d7cf455 (127.7.137.3:37593)
I20260619 02:41:09.050065 14421 ts_manager.cc:194] Registered new tserver with Master: 608d55b69efe446691a329e44d7cf455 (127.7.137.3:37593)
I20260619 02:41:09.050282 13824 ts_manager.cc:194] Re-registered known tserver with Master: 608d55b69efe446691a329e44d7cf455 (127.7.137.3:37593)
I20260619 02:41:09.050751 13824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:38989
I20260619 02:41:09.051595 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.037914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.7.137.61:33765)
0619 02:41:09.038103 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.038106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.038464 (+ 358us) server_negotiation.cc:407] Connection header received
0619 02:41:09.038485 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.038486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.038497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.038550 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.040222 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.040952 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.043639 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.043928 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.044000 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.046514 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.046514 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.046515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.046521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.048820 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.050010 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.050011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.050011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.050069 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.050982 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.050984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.050984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.051123 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.051216 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.051399 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.051479 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":58,"threads_started":1}
I20260619 02:41:09.052274 15072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.037442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.7.137.62:34665)
0619 02:41:09.038180 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.038182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.038392 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:41:09.038413 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.038414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.038426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.038498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.039296 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.042205 (+ 2909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.043130 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.043506 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.043575 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.045482 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.045483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.045484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.045491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.051096 (+ 5605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.051477 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.051478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.051478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.051536 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.051727 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.051727 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.051728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.051883 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.051979 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.052081 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.052181 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":78,"threads_started":1}
I20260619 02:41:09.052763 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.037250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.7.137.60:39015)
0619 02:41:09.038805 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.038807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.038815 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:09.038835 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.038836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.038847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.038909 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.041086 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.041822 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.044052 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.044349 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.044416 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.048897 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.048898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.048898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.049149 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.051186 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.051899 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.051900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.051901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.051962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.052167 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.052167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.052168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.052296 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.052388 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.052560 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.052634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":51,"threads_started":1}
I20260619 02:41:09.053773 13824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:09.055339 13824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:09.056056 14571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: NON_VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: true } } }
I20260619 02:41:09.056151 14571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:09.057312 14571 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: attempting to promote NON_VOTER 59fbc2207cb14cb5b357213c604474bb to VOTER
I20260619 02:41:09.057439 14571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:09.057897 14431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 02:41:09.058125 14616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 LEARNER]: Refusing update from remote peer 08d4a8dac973410cbc95a25d55480d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 02:41:09.058205 14382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 02:41:09.058531 14382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 02:41:09.058895 14383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 59fbc2207cb14cb5b357213c604474bb (127.7.137.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.059034 14431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 59fbc2207cb14cb5b357213c604474bb (127.7.137.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.059286 14616 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 59fbc2207cb14cb5b357213c604474bb (127.7.137.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.059871 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.059862 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.059966 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:09.059985 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:09.060015 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.060070 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:09.060021 14570 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.060132 14570 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:41:09.060285 14646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08d4a8dac973410cbc95a25d55480d02" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:09.060365 14646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:41:09.064996 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41261 (local address 127.0.0.1:42280)
0619 02:41:09.061360 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.061369 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.061411 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.061698 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.061699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.061702 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:09.061756 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.061758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.062738 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.062739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.063690 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.063692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.064705 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.064706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.064811 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.064815 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.064872 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.064911 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":6,"thread_start_us":86,"threads_started":1}
I20260619 02:41:09.065025 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.7.137.2:41261)
0619 02:41:09.061472 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.061478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.061489 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:41:09.061531 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.061534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.061551 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.061629 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:09.061822 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.062657 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.063783 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.064529 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.064598 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.064675 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.064888 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.064931 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260619 02:41:09.065274 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:37593 (local address 127.0.0.1:52118)
0619 02:41:09.061149 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.061160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.061197 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.061821 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.061822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.061823 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:09.061856 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.061857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.063441 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.063442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.064192 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.064194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.065107 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.065108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.065170 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.065173 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.065208 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.065233 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:41:09.065546 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.7.137.3:37593)
0619 02:41:09.061665 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.061668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.061677 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:09.061708 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.061710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.061724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.061926 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:09.061974 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.063344 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.064276 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.064998 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.065322 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.065394 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.065420 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.065461 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":60,"threads_started":1}
I20260619 02:41:09.065841 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45719 (local address 127.0.0.1:52998)
0619 02:41:09.061495 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.061504 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.061547 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.061722 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.061723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.061725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:09.061791 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.061793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.063385 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.063386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.064545 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.064547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.065598 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.065599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.065681 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.065685 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.065739 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.065777 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":159,"thread_start_us":60,"threads_started":1}
I20260619 02:41:09.065872 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.061205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.7.137.1:45719)
0619 02:41:09.061486 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.061490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.061539 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:41:09.061614 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.061616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.061628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.061724 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:09.061858 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.063313 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.064629 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.065415 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.065495 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.065565 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.065755 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.065793 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":85,"threads_started":1}
I20260619 02:41:09.066167 14993 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 (DEFAULT_TABLE table=test-table [id=fe91ffb26fbf48fca9e7b698d368e2ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:09.066165 14857 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 (DEFAULT_TABLE table=test-table [id=fe91ffb26fbf48fca9e7b698d368e2ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:09.066501 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae43fc666ad4ff38b7f1fc930c7d783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:09.066510 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae43fc666ad4ff38b7f1fc930c7d783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:09.069945 15090 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Bootstrap starting.
I20260619 02:41:09.070601 15090 tablet_bootstrap.cc:654] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:09.070555 14720 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 (DEFAULT_TABLE table=test-table [id=fe91ffb26fbf48fca9e7b698d368e2ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:09.070840 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae43fc666ad4ff38b7f1fc930c7d783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:09.070907 15090 log.cc:824] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:09.071205 15089 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Bootstrap starting.
I20260619 02:41:09.071684 15090 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: No bootstrap required, opened a new log
I20260619 02:41:09.071810 15089 tablet_bootstrap.cc:654] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:09.071810 15090 ts_tablet_manager.cc:1397] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:09.072120 15089 log.cc:824] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:09.073231 15089 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: No bootstrap required, opened a new log
I20260619 02:41:09.073302 15089 ts_tablet_manager.cc:1397] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:09.074011 15093 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Bootstrap starting.
I20260619 02:41:09.074616 15090 raft_consensus.cc:348] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.074806 15093 tablet_bootstrap.cc:654] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:09.074831 15090 raft_consensus.cc:374] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:09.074872 15090 raft_consensus.cc:729] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608d55b69efe446691a329e44d7cf455, State: Initialized, Role: FOLLOWER
I20260619 02:41:09.075001 15090 consensus_queue.cc:260] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.075133 15093 log.cc:824] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:09.075282 15090 ts_tablet_manager.cc:1428] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:41:09.075407 15059 heartbeater.cc:503] Master 127.7.137.62:34665 was elected leader, sending a full tablet report...
I20260619 02:41:09.075659 15089 raft_consensus.cc:348] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.075815 15089 raft_consensus.cc:374] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:09.075851 15089 raft_consensus.cc:729] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf36019923a9410f8df8b695d1faec65, State: Initialized, Role: FOLLOWER
I20260619 02:41:09.075985 15089 consensus_queue.cc:260] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.076187 15093 tablet_bootstrap.cc:492] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: No bootstrap required, opened a new log
I20260619 02:41:09.076253 15089 ts_tablet_manager.cc:1428] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:09.076287 15093 ts_tablet_manager.cc:1397] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:09.076346 14923 heartbeater.cc:503] Master 127.7.137.62:34665 was elected leader, sending a full tablet report...
I20260619 02:41:09.078989 15093 raft_consensus.cc:348] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.079107 15093 raft_consensus.cc:374] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:09.079128 15093 raft_consensus.cc:729] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14589babd74421ca6286633352959fc, State: Initialized, Role: FOLLOWER
I20260619 02:41:09.079253 15093 consensus_queue.cc:260] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.079528 14786 heartbeater.cc:503] Master 127.7.137.62:34665 was elected leader, sending a full tablet report...
I20260619 02:41:09.079523 15093 ts_tablet_manager.cc:1428] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:09.102013 15094 raft_consensus.cc:482] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:09.102108 15094 raft_consensus.cc:504] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.102397 15094 leader_election.cc:288] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b14589babd74421ca6286633352959fc (127.7.137.1:45719), cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:09.105188 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.102585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:39847 (local address 127.7.137.1:45719)
0619 02:41:09.102634 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.102636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.102645 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:09.102727 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.102728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.102739 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.102784 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:09.102951 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.103426 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104376 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.104834 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104918 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.104928 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.105073 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.105115 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:09.105204 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.102530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45719 (local address 127.7.137.3:39847)
0619 02:41:09.102584 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.102596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.102650 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.102796 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.102797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.102798 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:09.102869 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.102870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.103509 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.103510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.104280 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.104282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.104917 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.104998 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.105001 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.105066 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.105103 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":31}
I20260619 02:41:09.105221 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.102745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:49489 (local address 127.7.137.2:41261)
0619 02:41:09.102811 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.102813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.102819 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:41:09.102839 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.102839 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.102848 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.102912 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:09.103059 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.103507 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104404 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.104811 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104912 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.104924 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.105115 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.105152 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:41:09.105278 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.102568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41261 (local address 127.7.137.3:49489)
0619 02:41:09.102632 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:09.102642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:09.102681 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:09.102915 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:09.102916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:09.102919 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:09.102982 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.102984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.103587 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.103588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.104285 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:09.104287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.104948 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.104949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:09.105038 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.105041 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:09.105110 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:41:09.105147 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:41:09.105574 14740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae43fc666ad4ff38b7f1fc930c7d783" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14589babd74421ca6286633352959fc" is_pre_election: true
I20260619 02:41:09.105734 14740 raft_consensus.cc:2454] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 0.
I20260619 02:41:09.105789 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae43fc666ad4ff38b7f1fc930c7d783" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf36019923a9410f8df8b695d1faec65" is_pre_election: true
I20260619 02:41:09.105943 14877 raft_consensus.cc:2454] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 0.
I20260619 02:41:09.105950 14947 leader_election.cc:302] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 608d55b69efe446691a329e44d7cf455, b14589babd74421ca6286633352959fc; no voters:
I20260619 02:41:09.106078 15094 raft_consensus.cc:2781] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:09.106134 15094 raft_consensus.cc:482] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:09.106158 15094 raft_consensus.cc:3037] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:09.106891 15094 raft_consensus.cc:504] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.107030 15094 leader_election.cc:288] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 election: Requested vote from peers b14589babd74421ca6286633352959fc (127.7.137.1:45719), cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:09.107197 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae43fc666ad4ff38b7f1fc930c7d783" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf36019923a9410f8df8b695d1faec65"
I20260619 02:41:09.107295 14877 raft_consensus.cc:3037] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:09.107970 14877 raft_consensus.cc:2454] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 1.
I20260619 02:41:09.108094 14740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae43fc666ad4ff38b7f1fc930c7d783" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14589babd74421ca6286633352959fc"
I20260619 02:41:09.108170 14948 leader_election.cc:302] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 608d55b69efe446691a329e44d7cf455, cf36019923a9410f8df8b695d1faec65; no voters:
I20260619 02:41:09.108170 14740 raft_consensus.cc:3037] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:09.108280 15094 raft_consensus.cc:2781] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:09.108467 15094 raft_consensus.cc:686] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 1 LEADER]: Becoming Leader. State: Replica: 608d55b69efe446691a329e44d7cf455, State: Running, Role: LEADER
I20260619 02:41:09.108572 15094 consensus_queue.cc:237] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } }
I20260619 02:41:09.109364 13824 catalog_manager.cc:5495] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 608d55b69efe446691a329e44d7cf455 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "608d55b69efe446691a329e44d7cf455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:09.109506 14740 raft_consensus.cc:2454] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 1.
I20260619 02:41:09.124397 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.122200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.7.137.3:37593)
0619 02:41:09.122252 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.122254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.122261 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:09.122297 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.122297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.122320 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.122382 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:09.122552 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.123057 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.123750 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.123921 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.123969 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.124100 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:09.124195 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:09.124251 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.124281 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.124295 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:41:09.126173 14877 raft_consensus.cc:1261] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Refusing update from remote peer 608d55b69efe446691a329e44d7cf455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:09.126189 14740 raft_consensus.cc:1261] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Refusing update from remote peer 608d55b69efe446691a329e44d7cf455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:09.126479 15098 consensus_queue.cc:1035] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:09.126601 15094 consensus_queue.cc:1035] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:09.129730 15103 mvcc.cc:205] Tried to move back new op lower bound from 7298403815938125824 to 7298403815869493248. Current Snapshot: MvccSnapshot[applied={T|T < 7298403815938125824}]
I20260619 02:41:09.129781 15102 mvcc.cc:205] Tried to move back new op lower bound from 7298403815938125824 to 7298403815869493248. Current Snapshot: MvccSnapshot[applied={T|T < 7298403815938125824}]
I20260619 02:41:09.131637 13824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:09.131729 13824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:09.132992 13824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08d4a8dac973410cbc95a25d55480d02: Sending DeleteTablet for 3 replicas of tablet 9ae43fc666ad4ff38b7f1fc930c7d783
I20260619 02:41:09.133322 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:09 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:52118
I20260619 02:41:09.133303 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:09 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42280
I20260619 02:41:09.133525 15111 tablet_replica.cc:330] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: stopping tablet replica
I20260619 02:41:09.133630 14720 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae43fc666ad4ff38b7f1fc930c7d783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:09 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:52998
I20260619 02:41:09.133667 15112 tablet_replica.cc:330] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: stopping tablet replica
I20260619 02:41:09.133855 15112 raft_consensus.cc:2229] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:09.133982 15111 raft_consensus.cc:2229] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:09.134130 15111 raft_consensus.cc:2258] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:09.134253 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 13792
I20260619 02:41:09.134282 15112 raft_consensus.cc:2258] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:09.135015 15112 ts_tablet_manager.cc:1905] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:09.136657 15111 ts_tablet_manager.cc:1905] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:09.137885 15111 ts_tablet_manager.cc:1918] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:09.137972 15111 log.cc:1196] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/9ae43fc666ad4ff38b7f1fc930c7d783
I20260619 02:41:09.138228 15111 ts_tablet_manager.cc:1939] T 9ae43fc666ad4ff38b7f1fc930c7d783 P cf36019923a9410f8df8b695d1faec65: Deleting consensus metadata
W20260619 02:41:09.141271 14810 connection.cc:535] server connection from 127.0.0.1:42280 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:41:09.142125 15112 ts_tablet_manager.cc:1918] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:09.142206 15112 log.cc:1196] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/9ae43fc666ad4ff38b7f1fc930c7d783
I20260619 02:41:09.142511 15112 ts_tablet_manager.cc:1939] T 9ae43fc666ad4ff38b7f1fc930c7d783 P 608d55b69efe446691a329e44d7cf455: Deleting consensus metadata
W20260619 02:41:09.142861 14949 connection.cc:441] server connection from 127.0.0.1:52118 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52118 (request call id 1) could send its response
I20260619 02:41:09.149870 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.134091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.7.137.60:39015)
0619 02:41:09.134131 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.134133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.140917 (+ 6784us) server_negotiation.cc:407] Connection header received
0619 02:41:09.140994 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.140995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.141010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.141066 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.141958 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.142676 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.143766 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.144108 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.144147 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.145757 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.145758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.145758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.145764 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.147955 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.148950 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.148951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.148951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.149006 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.149237 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.149237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.149238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.149368 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.149463 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.149654 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.149755 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:09.151192 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.142980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.7.137.61:33765)
0619 02:41:09.143026 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.143028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.143340 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:41:09.143407 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.143408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.143421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.143475 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.144141 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.144874 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.145963 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.146308 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.146354 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.147835 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.147837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.147837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.147844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.149876 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.150386 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.150387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.150388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.150442 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.150649 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.150649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.150650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.150767 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.150833 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.151019 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.151099 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:41:09.151309 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:09.143183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.7.137.60:39015)
0619 02:41:09.143221 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:09.143222 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:09.143491 (+ 269us) server_negotiation.cc:407] Connection header received
0619 02:41:09.143583 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:09.143584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:09.143594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:09.143624 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:09.144812 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.145519 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.146552 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:09.146830 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:09.146869 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:09.148292 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:09.148293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:09.148293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:09.148302 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:09.149948 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.150529 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.150530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.150531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.150582 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:09.150792 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:09.150793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:09.150794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:09.150905 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:09.151004 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:09.151121 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:41:09.151193 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:09.152884 15113 tablet_replica.cc:330] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: stopping tablet replica
I20260619 02:41:09.153021 15113 raft_consensus.cc:2229] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:09.153173 15113 raft_consensus.cc:2258] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:09.153599 15113 ts_tablet_manager.cc:1905] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:09.155056 15113 ts_tablet_manager.cc:1918] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:09.155143 15113 log.cc:1196] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/9ae43fc666ad4ff38b7f1fc930c7d783
I20260619 02:41:09.155428 15113 ts_tablet_manager.cc:1939] T 9ae43fc666ad4ff38b7f1fc930c7d783 P b14589babd74421ca6286633352959fc: Deleting consensus metadata
W20260619 02:41:09.155797 14676 connection.cc:441] server connection from 127.0.0.1:52998 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52998 (request call id 1) could send its response
I20260619 02:41:10.131731 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.131493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:47915)
0619 02:41:10.131609 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.131686 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
I20260619 02:41:10.131752 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.131492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:50017)
0619 02:41:10.131636 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.131700 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260619 02:41:10.131945 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.131673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:56677)
0619 02:41:10.131799 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.131910 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":46,"threads_started":1}
W20260619 02:41:10.132032 15059 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:34665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
W20260619 02:41:10.132071 14923 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:34665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
W20260619 02:41:10.132071 14786 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:34665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.633196 15126 raft_consensus.cc:482] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 08d4a8dac973410cbc95a25d55480d02)
I20260619 02:41:10.633189 15125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 08d4a8dac973410cbc95a25d55480d02)
I20260619 02:41:10.633340 15126 raft_consensus.cc:504] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:10.633342 15125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:10.633632 15125 leader_election.cc:288] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), e15336f0c7684ca9a202817ec355dea5 (127.7.137.61:33765)
I20260619 02:41:10.633910 15126 leader_election.cc:288] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015)
I20260619 02:41:10.633955 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.633714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40998)
0619 02:41:10.633857 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.633907 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":53,"threads_started":1}
I20260619 02:41:10.633996 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.633715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:40986)
0619 02:41:10.633878 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.633913 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":56,"threads_started":1}
W20260619 02:41:10.634094 14408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
W20260619 02:41:10.634085 14594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.634488 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.634405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41000)
0619 02:41:10.634421 (+ 16us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.634459 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4}
W20260619 02:41:10.634626 14408 leader_election.cc:334] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.634688 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.634581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41012)
0619 02:41:10.634633 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.634663 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:41:10.634788 14594 leader_election.cc:334] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.636895 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.633744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39015 (local address 127.0.0.1:55038)
0619 02:41:10.634275 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.634284 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:10.634333 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:10.634523 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:10.634524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:10.634528 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:10.634573 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:10.634574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.635302 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.635303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:10.636081 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:10.636083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.636644 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:10.636699 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:10.636705 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:41:10.636777 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:41:10.636817 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":39,"threads_started":1}
I20260619 02:41:10.636987 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.633854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33765 (local address 127.0.0.1:45410)
0619 02:41:10.634020 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.634030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:10.634086 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:10.634340 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:10.634341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:10.634344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:10.634400 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:10.634401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.635361 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.635361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:10.636228 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:10.636230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.636779 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:10.636837 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:10.636840 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:10.636896 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:41:10.636931 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":31,"threads_started":1}
I20260619 02:41:10.637094 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.634041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.7.137.61:33765)
0619 02:41:10.634163 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:10.634165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:10.634174 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:10.634198 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:10.634199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:10.634210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:10.634286 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:10.634489 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.635286 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636307 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.636680 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636780 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:10.636789 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:10.636912 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 02:41:10.637035 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":40,"threads_started":1}
I20260619 02:41:10.637212 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.633890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.7.137.60:39015)
0619 02:41:10.634328 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:10.634330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:10.634338 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:10.634396 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:10.634397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:10.634406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:10.634479 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:10.634646 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.635226 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636153 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:10.636574 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:10.636624 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:10.636633 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:10.636806 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:41:10.636844 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":85,"threads_started":1}
I20260619 02:41:10.637264 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59fbc2207cb14cb5b357213c604474bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e15336f0c7684ca9a202817ec355dea5" is_pre_election: true
I20260619 02:41:10.637382 14431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59fbc2207cb14cb5b357213c604474bb in term 1.
I20260619 02:41:10.637450 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e15336f0c7684ca9a202817ec355dea5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59fbc2207cb14cb5b357213c604474bb" is_pre_election: true
I20260619 02:41:10.637537 14594 leader_election.cc:302] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb, e15336f0c7684ca9a202817ec355dea5; no voters: 08d4a8dac973410cbc95a25d55480d02
I20260619 02:41:10.637537 14616 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e15336f0c7684ca9a202817ec355dea5 in term 1.
I20260619 02:41:10.637656 15125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:41:10.637703 15125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 08d4a8dac973410cbc95a25d55480d02)
I20260619 02:41:10.637735 15125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:41:10.637774 14405 leader_election.cc:302] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb, e15336f0c7684ca9a202817ec355dea5; no voters: 08d4a8dac973410cbc95a25d55480d02
I20260619 02:41:10.637877 15126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:41:10.637909 15126 raft_consensus.cc:482] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 08d4a8dac973410cbc95a25d55480d02)
I20260619 02:41:10.637939 15126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:41:10.638466 15125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:10.638593 15125 leader_election.cc:288] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 election: Requested vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), e15336f0c7684ca9a202817ec355dea5 (127.7.137.61:33765)
I20260619 02:41:10.638628 15126 raft_consensus.cc:504] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:10.638746 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.638667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41018)
0619 02:41:10.638696 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.638722 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 02:41:10.638805 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59fbc2207cb14cb5b357213c604474bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e15336f0c7684ca9a202817ec355dea5"
I20260619 02:41:10.638878 14431 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 59fbc2207cb14cb5b357213c604474bb in current term 2: Already voted for candidate e15336f0c7684ca9a202817ec355dea5 in this term.
I20260619 02:41:10.638980 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e15336f0c7684ca9a202817ec355dea5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59fbc2207cb14cb5b357213c604474bb"
I20260619 02:41:10.639057 14616 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e15336f0c7684ca9a202817ec355dea5 in current term 2: Already voted for candidate 59fbc2207cb14cb5b357213c604474bb in this term.
I20260619 02:41:10.639070 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.638996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41034)
0619 02:41:10.639021 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.639046 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 02:41:10.639127 14594 leader_election.cc:334] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.639169 14594 leader_election.cc:302] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb; no voters: 08d4a8dac973410cbc95a25d55480d02, e15336f0c7684ca9a202817ec355dea5
I20260619 02:41:10.638877 15126 leader_election.cc:288] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 election: Requested vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015)
I20260619 02:41:10.638948 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.638834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41030)
0619 02:41:10.638904 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.638929 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 02:41:10.639225 15125 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:41:10.639547 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:10.639464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:41050)
0619 02:41:10.639507 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:10.639531 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:41:10.639647 14408 leader_election.cc:334] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:10.639700 14408 leader_election.cc:302] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e15336f0c7684ca9a202817ec355dea5; no voters: 08d4a8dac973410cbc95a25d55480d02, 59fbc2207cb14cb5b357213c604474bb
I20260619 02:41:10.639777 15126 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:41:11.132618 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:11.132383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:55149)
0619 02:41:11.132510 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:11.132556 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":49,"threads_started":1}
I20260619 02:41:11.132632 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:11.132403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:41419)
0619 02:41:11.132558 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:11.132593 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260619 02:41:11.132885 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:11.132340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:45041)
0619 02:41:11.132788 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:11.132828 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":94,"threads_started":1}
I20260619 02:41:12.133472 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.133060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:54745)
0619 02:41:12.133372 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.133413 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":62,"threads_started":1}
I20260619 02:41:12.133472 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.133247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:47555)
0619 02:41:12.133376 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.133419 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260619 02:41:12.133716 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.133260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:39793)
0619 02:41:12.133633 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.133669 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":50,"threads_started":1}
I20260619 02:41:12.239071 15141 raft_consensus.cc:482] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:12.239200 15141 raft_consensus.cc:504] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:12.239506 15141 leader_election.cc:288] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015)
I20260619 02:41:12.239713 15142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:12.239734 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e15336f0c7684ca9a202817ec355dea5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59fbc2207cb14cb5b357213c604474bb" is_pre_election: true
I20260619 02:41:12.239826 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.239623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60744)
0619 02:41:12.239733 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.239774 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260619 02:41:12.239820 15142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:12.239879 14616 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e15336f0c7684ca9a202817ec355dea5 in term 2.
I20260619 02:41:12.240038 14405 leader_election.cc:302] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb, e15336f0c7684ca9a202817ec355dea5; no voters:
I20260619 02:41:12.240137 15141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 02:41:12.240185 15141 raft_consensus.cc:482] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:12.240181 15142 leader_election.cc:288] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), e15336f0c7684ca9a202817ec355dea5 (127.7.137.61:33765)
I20260619 02:41:12.240206 15141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:41:12.240376 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.240195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60760)
0619 02:41:12.240300 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.240334 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":48,"threads_started":1}
I20260619 02:41:12.240410 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59fbc2207cb14cb5b357213c604474bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e15336f0c7684ca9a202817ec355dea5" is_pre_election: true
I20260619 02:41:12.240453 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.240154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60750)
0619 02:41:12.240398 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.240425 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
W20260619 02:41:12.240530 14408 leader_election.cc:334] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:12.240998 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.240877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60770)
0619 02:41:12.240950 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.240981 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 02:41:12.241106 14594 leader_election.cc:334] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:12.241178 15141 raft_consensus.cc:504] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:12.241293 14431 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 59fbc2207cb14cb5b357213c604474bb in current term 3: Already voted for candidate e15336f0c7684ca9a202817ec355dea5 in this term.
I20260619 02:41:12.241324 15141 leader_election.cc:288] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 election: Requested vote from peers 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665), 59fbc2207cb14cb5b357213c604474bb (127.7.137.60:39015)
I20260619 02:41:12.241457 14594 leader_election.cc:302] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb; no voters: 08d4a8dac973410cbc95a25d55480d02, e15336f0c7684ca9a202817ec355dea5
I20260619 02:41:12.241472 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e15336f0c7684ca9a202817ec355dea5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59fbc2207cb14cb5b357213c604474bb"
I20260619 02:41:12.241583 14616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:41:12.241683 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.241388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60776)
0619 02:41:12.241622 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.241653 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260619 02:41:12.241990 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.241897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60788)
0619 02:41:12.241942 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.241971 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:41:12.242070 14408 leader_election.cc:334] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
I20260619 02:41:12.242465 14616 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e15336f0c7684ca9a202817ec355dea5 in term 3.
I20260619 02:41:12.242556 15142 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260619 02:41:12.242664 14405 leader_election.cc:302] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59fbc2207cb14cb5b357213c604474bb, e15336f0c7684ca9a202817ec355dea5; no voters: 08d4a8dac973410cbc95a25d55480d02
I20260619 02:41:12.242762 15141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:41:12.242872 15141 raft_consensus.cc:686] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [term 3 LEADER]: Becoming Leader. State: Replica: e15336f0c7684ca9a202817ec355dea5, State: Running, Role: LEADER
I20260619 02:41:12.242942 15141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } }
I20260619 02:41:12.243287 15147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e15336f0c7684ca9a202817ec355dea5. Latest consensus state: current_term: 3 leader_uuid: "e15336f0c7684ca9a202817ec355dea5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:12.243372 15147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:12.243530 15149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:12.243944 15149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:12.244004 15149 catalog_manager.cc:1259] Loaded cluster ID: 65ec451cfc7543abbae75321a7541b09
I20260619 02:41:12.244055 15149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:12.244236 15149 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:12.244321 15149 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Loaded TSK: 0
I20260619 02:41:12.244469 15149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:12.311764 14421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:12.313596 14421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:12.314654 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.314554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60808)
0619 02:41:12.314589 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.314621 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 02:41:12.314698 14616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [term 3 FOLLOWER]: Refusing update from remote peer e15336f0c7684ca9a202817ec355dea5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 02:41:12.314922 15147 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 02:41:12.315021 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.314920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.0.0.1:60810)
0619 02:41:12.314976 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.315001 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:41:12.315111 14408 consensus_peers.cc:487] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 -> Peer 08d4a8dac973410cbc95a25d55480d02 (127.7.137.62:34665): Couldn't send request to peer 08d4a8dac973410cbc95a25d55480d02. Status: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:41:12.315578 15142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e15336f0c7684ca9a202817ec355dea5. Latest consensus state: current_term: 3 leader_uuid: "e15336f0c7684ca9a202817ec355dea5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:12.315680 15142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:41:12.315982 15150 mvcc.cc:205] Tried to move back new op lower bound from 7298403828998901760 to 7298403828707692544. Current Snapshot: MvccSnapshot[applied={T|T < 7298403828998901760}]
I20260619 02:41:12.316112 15151 mvcc.cc:205] Tried to move back new op lower bound from 7298403828998901760 to 7298403828707692544. Current Snapshot: MvccSnapshot[applied={T|T < 7298403828998901760}]
I20260619 02:41:12.316223 15142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e15336f0c7684ca9a202817ec355dea5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:12.316362 15142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fbc2207cb14cb5b357213c604474bb [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:41:12.316421 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e15336f0c7684ca9a202817ec355dea5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "08d4a8dac973410cbc95a25d55480d02" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 34665 } } peers { permanent_uuid: "e15336f0c7684ca9a202817ec355dea5" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33765 } attrs { promote: false } } peers { permanent_uuid: "59fbc2207cb14cb5b357213c604474bb" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39015 } attrs { promote: false } } }
I20260619 02:41:12.316506 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:12.320957 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.317970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45719 (local address 127.0.0.1:49308)
0619 02:41:12.318008 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.318018 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:12.318064 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:12.318480 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:12.318481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:12.318482 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:12.318526 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.318527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.319210 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.319211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.319940 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.319942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.320695 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.320696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.320787 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.320792 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:41:12.320856 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:41:12.320896 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:41:12.320981 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.318048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.7.137.1:45719)
0619 02:41:12.318353 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:12.318355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:12.318363 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:12.318387 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:12.318388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:12.318398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:12.318476 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:12.318610 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.319127 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.320031 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.320614 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.320683 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.320696 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:12.320865 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:41:12.320908 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":68,"threads_started":1}
I20260619 02:41:12.321375 14720 tablet_service.cc:1467] Processing CreateTablet for tablet bfca95985cb244f4adf4300bde459da9 (DEFAULT_TABLE table=test-table [id=6f7ea27966fe45268476dcc7f03a8c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:12.321517 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfca95985cb244f4adf4300bde459da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:12.321740 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.318100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:37593 (local address 127.0.0.1:44834)
0619 02:41:12.318289 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.318298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:12.318354 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:12.319316 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:12.319317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:12.319319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:12.319359 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.319360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.319958 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.319960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.320699 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.320700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.321519 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.321520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.321582 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.321586 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:12.321636 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:41:12.321674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260619 02:41:12.321740 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.318202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.7.137.3:37593)
0619 02:41:12.318304 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:12.318306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:12.318345 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:41:12.319207 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:12.319208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:12.319218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:12.319301 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:12.319409 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.319878 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.320806 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.321421 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.321479 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.321490 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:12.321640 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:41:12.321675 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260619 02:41:12.321956 14993 tablet_service.cc:1467] Processing CreateTablet for tablet bfca95985cb244f4adf4300bde459da9 (DEFAULT_TABLE table=test-table [id=6f7ea27966fe45268476dcc7f03a8c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:12.322068 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfca95985cb244f4adf4300bde459da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:12.323077 15161 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: Bootstrap starting.
I20260619 02:41:12.323092 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.318019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41261 (local address 127.0.0.1:42090)
0619 02:41:12.318197 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.318207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:12.318258 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:12.318468 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:12.318469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:12.318471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:12.318522 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.318524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.320050 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.320051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.321520 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:12.321522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.322769 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.322770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:12.322883 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.322887 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:12.322997 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:41:12.323031 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
I20260619 02:41:12.323340 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.318191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.7.137.2:41261)
0619 02:41:12.318285 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:12.318287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:12.318296 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:12.318324 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:12.318325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:12.318336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:12.318424 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:12.318601 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.319127 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.321633 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.322589 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.322665 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.322677 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:12.323206 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 02:41:12.323260 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":50,"threads_started":1}
I20260619 02:41:12.323539 14857 tablet_service.cc:1467] Processing CreateTablet for tablet bfca95985cb244f4adf4300bde459da9 (DEFAULT_TABLE table=test-table [id=6f7ea27966fe45268476dcc7f03a8c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:12.323582 15162 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Bootstrap starting.
I20260619 02:41:12.323644 15161 tablet_bootstrap.cc:654] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:12.323653 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfca95985cb244f4adf4300bde459da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:12.323935 15162 tablet_bootstrap.cc:654] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:12.324220 15161 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: No bootstrap required, opened a new log
I20260619 02:41:12.324283 15161 ts_tablet_manager.cc:1397] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:12.324433 15161 raft_consensus.cc:348] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.324499 15162 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: No bootstrap required, opened a new log
I20260619 02:41:12.324512 15161 raft_consensus.cc:374] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:12.324535 15161 raft_consensus.cc:729] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14589babd74421ca6286633352959fc, State: Initialized, Role: FOLLOWER
I20260619 02:41:12.324541 15162 ts_tablet_manager.cc:1397] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:12.324613 15161 consensus_queue.cc:260] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.324654 15162 raft_consensus.cc:348] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.324760 15162 raft_consensus.cc:374] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:12.324793 15162 raft_consensus.cc:729] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608d55b69efe446691a329e44d7cf455, State: Initialized, Role: FOLLOWER
I20260619 02:41:12.324826 15161 ts_tablet_manager.cc:1428] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:12.324863 15162 consensus_queue.cc:260] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.325033 15162 ts_tablet_manager.cc:1428] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:12.325126 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.325021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:45889)
0619 02:41:12.325074 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.325102 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 02:41:12.325325 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.325188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:41209)
0619 02:41:12.325270 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.325296 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 02:41:12.325507 15167 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Bootstrap starting.
I20260619 02:41:12.325687 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.325602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:45253)
0619 02:41:12.325641 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.325664 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 02:41:12.325822 15167 tablet_bootstrap.cc:654] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:12.326167 15061 heartbeater.cc:503] Master 127.7.137.61:33765 was elected leader, sending a full tablet report...
I20260619 02:41:12.326342 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.326211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:43169)
0619 02:41:12.326290 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.326319 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 02:41:12.326552 15167 tablet_bootstrap.cc:492] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: No bootstrap required, opened a new log
I20260619 02:41:12.326606 15167 ts_tablet_manager.cc:1397] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:12.326730 15167 raft_consensus.cc:348] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.326781 15167 raft_consensus.cc:374] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:12.326793 15167 raft_consensus.cc:729] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf36019923a9410f8df8b695d1faec65, State: Initialized, Role: FOLLOWER
I20260619 02:41:12.326851 15167 consensus_queue.cc:260] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.327018 15167 ts_tablet_manager.cc:1428] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:12.327286 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.327173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:42419)
0619 02:41:12.327241 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.327264 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 02:41:12.327530 14925 heartbeater.cc:503] Master 127.7.137.61:33765 was elected leader, sending a full tablet report...
I20260619 02:41:12.327584 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.327458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:53745)
0619 02:41:12.327494 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.327543 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 02:41:12.391631 14926 tablet.cc:2369] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:41:12.490904 14789 tablet.cc:2369] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:41:12.536204 15062 tablet.cc:2369] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:12.646449 15166 raft_consensus.cc:482] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:12.646565 15166 raft_consensus.cc:504] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.646780 15166 leader_election.cc:288] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b14589babd74421ca6286633352959fc (127.7.137.1:45719), cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:12.647003 14740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfca95985cb244f4adf4300bde459da9" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14589babd74421ca6286633352959fc" is_pre_election: true
I20260619 02:41:12.647030 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfca95985cb244f4adf4300bde459da9" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf36019923a9410f8df8b695d1faec65" is_pre_election: true
I20260619 02:41:12.647104 14740 raft_consensus.cc:2454] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 0.
I20260619 02:41:12.647126 14877 raft_consensus.cc:2454] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 0.
I20260619 02:41:12.647297 14948 leader_election.cc:302] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 608d55b69efe446691a329e44d7cf455, cf36019923a9410f8df8b695d1faec65; no voters:
I20260619 02:41:12.647395 15166 raft_consensus.cc:2781] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:12.647437 15166 raft_consensus.cc:482] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:12.647457 15166 raft_consensus.cc:3037] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:12.648321 15166 raft_consensus.cc:504] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.648468 15166 leader_election.cc:288] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 election: Requested vote from peers b14589babd74421ca6286633352959fc (127.7.137.1:45719), cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261)
I20260619 02:41:12.648655 14740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfca95985cb244f4adf4300bde459da9" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14589babd74421ca6286633352959fc"
I20260619 02:41:12.648768 14740 raft_consensus.cc:3037] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:12.648680 14877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfca95985cb244f4adf4300bde459da9" candidate_uuid: "608d55b69efe446691a329e44d7cf455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf36019923a9410f8df8b695d1faec65"
I20260619 02:41:12.648813 14877 raft_consensus.cc:3037] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:12.649387 14740 raft_consensus.cc:2454] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 1.
I20260619 02:41:12.649560 14877 raft_consensus.cc:2454] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 608d55b69efe446691a329e44d7cf455 in term 1.
I20260619 02:41:12.649597 14947 leader_election.cc:302] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 608d55b69efe446691a329e44d7cf455, b14589babd74421ca6286633352959fc; no voters:
I20260619 02:41:12.649699 15166 raft_consensus.cc:2781] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:12.649848 15166 raft_consensus.cc:686] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 1 LEADER]: Becoming Leader. State: Replica: 608d55b69efe446691a329e44d7cf455, State: Running, Role: LEADER
I20260619 02:41:12.649919 15166 consensus_queue.cc:237] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } }
I20260619 02:41:12.650267 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.650154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:48491)
0619 02:41:12.650194 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.650234 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 02:41:12.650645 14421 catalog_manager.cc:5495] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 608d55b69efe446691a329e44d7cf455 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "608d55b69efe446691a329e44d7cf455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "608d55b69efe446691a329e44d7cf455" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 37593 } health_report { overall_health: HEALTHY } } }
I20260619 02:41:12.672959 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.670804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.7.137.3:37593)
0619 02:41:12.670878 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:12.670880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:12.670890 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:12.670913 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:12.670914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:12.670924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:12.670970 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:12.671096 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.671588 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.672316 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:12.672492 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:12.672540 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:12.672648 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:12.672674 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:12.672712 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:12.672827 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:41:12.672846 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:41:12.673959 14740 raft_consensus.cc:1261] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Refusing update from remote peer 608d55b69efe446691a329e44d7cf455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:12.674024 14877 raft_consensus.cc:1261] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Refusing update from remote peer 608d55b69efe446691a329e44d7cf455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:12.674206 15166 consensus_queue.cc:1035] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b14589babd74421ca6286633352959fc" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:12.674327 15170 consensus_queue.cc:1035] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf36019923a9410f8df8b695d1faec65" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:12.675048 14788 heartbeater.cc:503] Master 127.7.137.61:33765 was elected leader, sending a full tablet report...
I20260619 02:41:12.675292 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.675114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:42491)
0619 02:41:12.675222 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.675257 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 02:41:12.675376 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.675216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:48465)
0619 02:41:12.675316 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.675345 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 02:41:12.676093 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.676010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:47753)
0619 02:41:12.676044 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.676066 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 02:41:12.675782 15173 mvcc.cc:205] Tried to move back new op lower bound from 7298403830470692864 to 7298403830374998016. Current Snapshot: MvccSnapshot[applied={T|T < 7298403830470692864}]
I20260619 02:41:12.676484 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.676350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.3:33187)
0619 02:41:12.676432 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.676460 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 02:41:12.676838 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.676687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.2:54725)
0619 02:41:12.676783 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.676808 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 02:41:12.677425 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.677328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:34665 (local address 127.7.137.1:45463)
0619 02:41:12.677366 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.677399 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 02:41:12.678402 14420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:12.678489 14420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:12.679736 14420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e15336f0c7684ca9a202817ec355dea5: Sending DeleteTablet for 3 replicas of tablet bfca95985cb244f4adf4300bde459da9
I20260619 02:41:12.680032 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet bfca95985cb244f4adf4300bde459da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:12 UTC) from {username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:42090
I20260619 02:41:12.680032 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet bfca95985cb244f4adf4300bde459da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:12 UTC) from {username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:44834
I20260619 02:41:12.680064 14720 tablet_service.cc:1514] Processing DeleteTablet for tablet bfca95985cb244f4adf4300bde459da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:12 UTC) from {username='oryx', principal='oryx/127.7.137.61@KRBTEST.COM'} at 127.0.0.1:49308
I20260619 02:41:12.680245 15184 tablet_replica.cc:330] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc: stopping tablet replica
I20260619 02:41:12.680235 15182 tablet_replica.cc:330] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: stopping tablet replica
I20260619 02:41:12.680294 15184 raft_consensus.cc:2229] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:12.680294 15182 raft_consensus.cc:2229] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:12.680339 15184 raft_consensus.cc:2258] T bfca95985cb244f4adf4300bde459da9 P b14589babd74421ca6286633352959fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:12.680341 15182 raft_consensus.cc:2258] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:12.680416 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14568
I20260619 02:41:12.680574 15183 tablet_replica.cc:330] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: stopping tablet replica
I20260619 02:41:12.680624 15183 raft_consensus.cc:2229] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:12.680672 15183 raft_consensus.cc:2258] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:12.681397 15182 ts_tablet_manager.cc:1905] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:12.681512 15183 ts_tablet_manager.cc:1905] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:12.683372 15183 ts_tablet_manager.cc:1918] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:12.683377 15182 ts_tablet_manager.cc:1918] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:12.683421 15183 log.cc:1196] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/bfca95985cb244f4adf4300bde459da9
I20260619 02:41:12.683421 15182 log.cc:1196] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/bfca95985cb244f4adf4300bde459da9
I20260619 02:41:12.683661 15182 ts_tablet_manager.cc:1939] T bfca95985cb244f4adf4300bde459da9 P cf36019923a9410f8df8b695d1faec65: Deleting consensus metadata
I20260619 02:41:12.683661 15183 ts_tablet_manager.cc:1939] T bfca95985cb244f4adf4300bde459da9 P 608d55b69efe446691a329e44d7cf455: Deleting consensus metadata
I20260619 02:41:12.684113 14405 catalog_manager.cc:4841] TS 608d55b69efe446691a329e44d7cf455 (127.7.137.3:37593): tablet bfca95985cb244f4adf4300bde459da9 (table test-table [id=6f7ea27966fe45268476dcc7f03a8c19]) successfully deleted
I20260619 02:41:12.684162 14407 catalog_manager.cc:4841] TS cf36019923a9410f8df8b695d1faec65 (127.7.137.2:41261): tablet bfca95985cb244f4adf4300bde459da9 (table test-table [id=6f7ea27966fe45268476dcc7f03a8c19]) successfully deleted
I20260619 02:41:12.685894 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:12.685777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45719 (local address 127.0.0.1:49312)
0619 02:41:12.685827 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:12.685856 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:41:12.686098 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14794
W20260619 02:41:12.686079 14406 catalog_manager.cc:4584] TS b14589babd74421ca6286633352959fc (127.7.137.1:45719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfca95985cb244f4adf4300bde459da9: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45719: connect: Connection refused (error 111)
I20260619 02:41:12.692082 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14930
I20260619 02:41:12.698009 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14288
I20260619 02:41:12.702893 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 14475
2026-06-19T02:41:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6402 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:12 dist-test-slave-pdnb krb5kdc[15187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:12 dist-test-slave-pdnb krb5kdc[15187](info): set up 2 sockets
Jun 19 02:41:12 dist-test-slave-pdnb krb5kdc[15187](info): commencing operation
krb5kdc: starting...
W20260619 02:41:13.753638 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:13 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab.
Entry for principal oryx/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab.
I20260619 02:41:13.867873 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:39009
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab
--principal=oryx/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:39009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:13.946416 15200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:13.946638 15200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:13.946673 15200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:13.947821 15200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:13.947908 15200 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:41:13.947932 15200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:13.947955 15200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:13.947983 15200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:13.948006 15200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:13.949597 15200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:39009
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.62.keytab
--principal=oryx/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:13.949894 15200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:13.950107 15200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:13.955193 15206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:13.955204 15200 server_base.cc:1034] running on GCE node
W20260619 02:41:13.955181 15208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:13.955181 15205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:13.955790 15200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:13.956135 15200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:13.957274 15200 hybrid_clock.cc:648] HybridClock initialized: now 1781836873957252 us; error 40 us; skew 500 ppm
Jun 19 02:41:13 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:13.959928 15200 init.cc:377] Logged in from keytab as oryx/127.7.137.62@KRBTEST.COM (short username oryx)
I20260619 02:41:13.960456 15200 webserver.cc:460] Webserver started at http://127.7.137.62:44411/ using document root <none> and password file <none>
I20260619 02:41:13.960780 15200 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:13.960857 15200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:13.961002 15200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:13.962157 15200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "ab0d250f686e48cea3c795082092ba7b"
format_stamp: "Formatted at 2026-06-19 02:41:13 on dist-test-slave-pdnb"
server_key: "bca84dea7a51672df9d303f906c25582"
server_key_iv: "87029a5f3bce2d23dd2c9a07095965b7"
server_key_version: "encryptionkey@0"
I20260619 02:41:13.962540 15200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "ab0d250f686e48cea3c795082092ba7b"
format_stamp: "Formatted at 2026-06-19 02:41:13 on dist-test-slave-pdnb"
server_key: "bca84dea7a51672df9d303f906c25582"
server_key_iv: "87029a5f3bce2d23dd2c9a07095965b7"
server_key_version: "encryptionkey@0"
I20260619 02:41:13.963922 15200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:13.964716 15215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:13.964941 15200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:13.965049 15200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "ab0d250f686e48cea3c795082092ba7b"
format_stamp: "Formatted at 2026-06-19 02:41:13 on dist-test-slave-pdnb"
server_key: "bca84dea7a51672df9d303f906c25582"
server_key_iv: "87029a5f3bce2d23dd2c9a07095965b7"
server_key_version: "encryptionkey@0"
I20260619 02:41:13.965140 15200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:13.977561 15200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:13.979504 15200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:13.979671 15200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:13.984613 15200 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:39009
I20260619 02:41:13.984660 15277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:39009 every 8 connection(s)
I20260619 02:41:13.985086 15200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:13.985747 15278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:13.988022 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Bootstrap starting.
I20260619 02:41:13.988649 15278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:13.988970 15278 log.cc:824] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:13.989753 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: No bootstrap required, opened a new log
I20260619 02:41:13.991858 15278 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } }
I20260619 02:41:13.992007 15278 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:13.992030 15278 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0d250f686e48cea3c795082092ba7b, State: Initialized, Role: FOLLOWER
I20260619 02:41:13.992223 15278 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } }
I20260619 02:41:13.992311 15278 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:13.992349 15278 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:13.992398 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15200
I20260619 02:41:13.992416 15278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:13.992491 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:13.992682 7716 external_mini_cluster.cc:1428] Setting key 968267c0507b4d07d3f929d32ce87fa8
I20260619 02:41:13.993186 15278 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } }
I20260619 02:41:13.993299 15278 leader_election.cc:302] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab0d250f686e48cea3c795082092ba7b; no voters:
I20260619 02:41:13.993461 15278 leader_election.cc:288] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:13.993492 15283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:13.993623 15283 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [term 1 LEADER]: Becoming Leader. State: Replica: ab0d250f686e48cea3c795082092ba7b, State: Running, Role: LEADER
I20260619 02:41:13.993716 15283 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } }
I20260619 02:41:13.993733 15278 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:13.994019 15283 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab0d250f686e48cea3c795082092ba7b. Latest consensus state: current_term: 1 leader_uuid: "ab0d250f686e48cea3c795082092ba7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } } }
I20260619 02:41:13.994108 15283 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:13.994028 15284 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab0d250f686e48cea3c795082092ba7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d250f686e48cea3c795082092ba7b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 39009 } } }
I20260619 02:41:13.994267 15284 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:13.994357 15287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:13.994858 15287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:13.996407 15287 catalog_manager.cc:1347] Generated new cluster ID: 8936bc9db1794bf4bd8b1a9bc9201cb1
I20260619 02:41:13.996459 15287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:41:13 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.003234 15287 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:14.003901 15287 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:14.004823 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:13.993253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.7.137.62:39009)
0619 02:41:13.993486 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:13.993496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:13.993537 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:41:13.993660 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:13.993667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:13.993722 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:13.994735 (+ 1013us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:13.994836 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:13.996284 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:13.997405 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:13.998041 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:13.998157 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.001058 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.001062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.001062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.001076 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.003154 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.003573 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.003576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.003577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.003681 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.003912 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.003913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.003914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.004174 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.004316 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.004446 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.004582 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260619 02:41:14.006232 15287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Generated new TSK 0
I20260619 02:41:14.006392 15287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
I20260619 02:41:14.032620 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:39009
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:14.112855 15304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:14.113050 15304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:14.113075 15304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:14.114207 15304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:14.114249 15304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:14.114269 15304 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:41:14.114286 15304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:14.115762 15304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:14.115998 15304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:14.116246 15304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:14.121176 15312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:14.121202 15310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:14.121219 15304 server_base.cc:1034] running on GCE node
W20260619 02:41:14.121170 15309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:14.121585 15304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:14.121831 15304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:14.122998 15304 hybrid_clock.cc:648] HybridClock initialized: now 1781836874122981 us; error 32 us; skew 500 ppm
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:14.125533 15304 init.cc:377] Logged in from keytab as oryx/127.7.137.1@KRBTEST.COM (short username oryx)
I20260619 02:41:14.125999 15304 webserver.cc:460] Webserver started at http://127.7.137.1:36321/ using document root <none> and password file <none>
I20260619 02:41:14.126250 15304 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:14.126320 15304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:14.126468 15304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:14.127571 15304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6561e88a6a7c062fda570bd80b749f65"
server_key_iv: "f183823f50345293c09fbaeddff212c6"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.127920 15304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6561e88a6a7c062fda570bd80b749f65"
server_key_iv: "f183823f50345293c09fbaeddff212c6"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.129274 15304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:14.129946 15319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.130118 15304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:14.130220 15304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6561e88a6a7c062fda570bd80b749f65"
server_key_iv: "f183823f50345293c09fbaeddff212c6"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.130292 15304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:14.152179 15304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:14.154064 15304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:14.154223 15304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:14.154767 15304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:14.154825 15304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.154866 15304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:14.154894 15304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.162091 15304 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:40975
I20260619 02:41:14.162220 15434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:40975 every 8 connection(s)
I20260619 02:41:14.162503 15304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:14.164017 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.155135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:52297 (local address 127.7.137.62:39009)
0619 02:41:14.155205 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.155207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.155425 (+ 218us) server_negotiation.cc:407] Connection header received
0619 02:41:14.155894 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.155895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.155907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.155985 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.156604 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.157252 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.158304 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.158516 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.158594 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.161137 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.161139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.161139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.161148 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.162850 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.163264 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.163264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.163265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.163319 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.163516 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.163517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.163517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.163618 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.163672 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.163877 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.163951 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:41:14.164165 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.155127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.1:52297)
0619 02:41:14.155287 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.155327 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.155793 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.156038 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.156041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.156265 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.156508 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.156513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.157345 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.157346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.158210 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.158212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.158599 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.158600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.158722 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.159197 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.159223 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.161039 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.162950 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.162956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.162958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.163198 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.163377 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.163378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.163378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.163465 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.163666 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.163669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.163773 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.163851 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.163968 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260619 02:41:14.164640 15435 heartbeater.cc:347] Connected to a master server at 127.7.137.62:39009
I20260619 02:41:14.164762 15435 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:14.164938 15435 heartbeater.cc:511] Master 127.7.137.62:39009 requested a full tablet report, sending...
I20260619 02:41:14.165498 15232 ts_manager.cc:194] Registered new tserver with Master: fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975)
I20260619 02:41:14.166326 15232 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:52297
I20260619 02:41:14.167294 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15304
I20260619 02:41:14.167397 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:14.167613 7716 external_mini_cluster.cc:1428] Setting key 4f4bc2a040562c05f07d21f2215eb54f
I20260619 02:41:14.171787 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.165706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:50587 (local address 127.7.137.62:39009)
0619 02:41:14.165745 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.165746 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.165876 (+ 130us) server_negotiation.cc:407] Connection header received
0619 02:41:14.165900 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.165900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.165910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.165982 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.166245 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.166871 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.167852 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.168133 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.168198 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.169155 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.169156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.169156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.169161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.170767 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.171151 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.171151 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.171152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.171197 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.171384 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.171384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.171384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.171473 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.171520 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.171680 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.171734 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:41:14.171926 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.165666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.1:50587)
0619 02:41:14.165799 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.165809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.165863 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.165983 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.165984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.166135 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.166184 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.166185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.166962 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.166963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.167771 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.167773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.168206 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.168207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.168273 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.168633 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.168638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.169067 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.170854 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.170855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.170855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.171076 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.171255 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.171256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.171257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.171327 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.171539 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.171540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.171589 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.171666 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.171777 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
I20260619 02:41:14.191298 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:39009
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:14.271209 15444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:14.271443 15444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:14.271481 15444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:14.272627 15444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:14.272694 15444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:14.272724 15444 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:41:14.272810 15444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:14.274343 15444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:14.274612 15444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:14.274844 15444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:14.279707 15452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:14.279707 15449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:14.279771 15450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:14.279983 15444 server_base.cc:1034] running on GCE node
I20260619 02:41:14.280189 15444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:14.280378 15444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:14.281529 15444 hybrid_clock.cc:648] HybridClock initialized: now 1781836874281508 us; error 34 us; skew 500 ppm
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:14.283946 15444 init.cc:377] Logged in from keytab as oryx/127.7.137.2@KRBTEST.COM (short username oryx)
I20260619 02:41:14.284310 15444 webserver.cc:460] Webserver started at http://127.7.137.2:44235/ using document root <none> and password file <none>
I20260619 02:41:14.284492 15444 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:14.284531 15444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:14.284616 15444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:14.285667 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "f12a7e570c7e489eaa6b0a1516b867c4"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "51ccabc38b27dca25422e61dee65ff65"
server_key_iv: "8686842ba144373fb4218779ced356e3"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.285956 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "f12a7e570c7e489eaa6b0a1516b867c4"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "51ccabc38b27dca25422e61dee65ff65"
server_key_iv: "8686842ba144373fb4218779ced356e3"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.287164 15444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:14.287798 15459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.287973 15444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:14.288077 15444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "f12a7e570c7e489eaa6b0a1516b867c4"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "51ccabc38b27dca25422e61dee65ff65"
server_key_iv: "8686842ba144373fb4218779ced356e3"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.288151 15444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:14.309003 15444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:14.311328 15444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:14.311504 15444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:14.312017 15444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:14.312080 15444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.312119 15444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:14.312163 15444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.318977 15444 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:45821
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.319051 15574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:45821 every 8 connection(s)
I20260619 02:41:14.319362 15444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:14.322296 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.312540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:37131 (local address 127.7.137.62:39009)
0619 02:41:14.312613 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.312615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.312842 (+ 227us) server_negotiation.cc:407] Connection header received
0619 02:41:14.313432 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.313433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.313448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.313532 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.314211 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.315064 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.316754 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.317086 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.317164 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.319513 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.319515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.319515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.319522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.320939 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.321347 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.321349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.321349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.321407 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.321661 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.321662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.321663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.321781 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.321857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.322123 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.322205 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:41:14.322481 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.312508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.2:37131)
0619 02:41:14.312686 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.312747 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.313288 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.313620 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.313624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.313865 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.314137 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.314142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.315179 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.315180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.316587 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.316590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.317185 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.317186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.317285 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.317818 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.317829 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.319407 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.321029 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.321036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.321038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.321256 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.321474 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.321475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.321476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.321592 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.321848 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.321854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.321993 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.322102 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.322257 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260619 02:41:14.322948 15575 heartbeater.cc:347] Connected to a master server at 127.7.137.62:39009
I20260619 02:41:14.323038 15575 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:14.323236 15575 heartbeater.cc:511] Master 127.7.137.62:39009 requested a full tablet report, sending...
I20260619 02:41:14.323625 15231 ts_manager.cc:194] Registered new tserver with Master: f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821)
I20260619 02:41:14.324136 15231 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:37131
I20260619 02:41:14.326207 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15444
I20260619 02:41:14.326364 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:14.326594 7716 external_mini_cluster.cc:1428] Setting key 7be681e9a10df6887e08cc37c44fd54f
I20260619 02:41:14.330116 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.324071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:38047 (local address 127.7.137.62:39009)
0619 02:41:14.324262 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.324264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.324271 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:14.324288 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.324288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.324298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.324345 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.324695 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.325259 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.325908 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.326190 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.326262 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.327237 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.327238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.327239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.327244 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.328906 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.329319 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.329320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.329320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.329375 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.329638 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.329639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.329639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.329775 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.329837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.329999 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.330058 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260619 02:41:14.330271 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.323989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.2:38047)
0619 02:41:14.324112 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.324122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.324166 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.324350 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.324351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.324553 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.324617 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.324620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.325348 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.325348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.325822 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.325823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.326280 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.326281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.326345 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.326770 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.326775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.327137 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.329002 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.329003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.329004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.329243 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.329441 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.329441 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.329442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.329509 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.329846 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.329847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.329893 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.329975 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.330089 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
I20260619 02:41:14.351042 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:39009
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:14.430757 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:14.430951 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:14.430974 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:14.432049 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:14.432083 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:14.432103 15584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:41:14.432121 15584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:14.433643 15584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.137.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:14.433827 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:14.433995 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:14.438978 15590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:14.439100 15584 server_base.cc:1034] running on GCE node
W20260619 02:41:14.438953 15589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:14.438968 15592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:14.439388 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:14.439632 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:14.440780 15584 hybrid_clock.cc:648] HybridClock initialized: now 1781836874440772 us; error 38 us; skew 500 ppm
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:14.443253 15584 init.cc:377] Logged in from keytab as oryx/127.7.137.3@KRBTEST.COM (short username oryx)
I20260619 02:41:14.443701 15584 webserver.cc:460] Webserver started at http://127.7.137.3:44759/ using document root <none> and password file <none>
I20260619 02:41:14.443931 15584 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:14.443996 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:14.444147 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:14.445231 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "b620b9073b8c490692cf6842cbe4836e"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6b9ce0acc54b54e935e40853820e83a8"
server_key_iv: "d4fe400d498668430ad5d27b22ad7d1f"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.445583 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "b620b9073b8c490692cf6842cbe4836e"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6b9ce0acc54b54e935e40853820e83a8"
server_key_iv: "d4fe400d498668430ad5d27b22ad7d1f"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.446913 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:14.447600 15599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.447767 15584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:14.447885 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "b620b9073b8c490692cf6842cbe4836e"
format_stamp: "Formatted at 2026-06-19 02:41:14 on dist-test-slave-pdnb"
server_key: "6b9ce0acc54b54e935e40853820e83a8"
server_key_iv: "d4fe400d498668430ad5d27b22ad7d1f"
server_key_version: "encryptionkey@0"
I20260619 02:41:14.447959 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:14.457470 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:14.459594 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:14.459745 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:14.460249 15584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:14.460309 15584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.460337 15584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:14.460356 15584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.467321 15584 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:41929
I20260619 02:41:14.467402 15714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:41929 every 8 connection(s)
I20260619 02:41:14.467753 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:14.470077 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.460569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:43955 (local address 127.7.137.62:39009)
0619 02:41:14.460631 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.460634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.460876 (+ 242us) server_negotiation.cc:407] Connection header received
0619 02:41:14.461377 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.461378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.461412 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.461470 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.462254 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.462751 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.463745 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.464023 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.464127 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.467471 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.467472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.467473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.467482 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.469002 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.469361 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.469361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.469362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.469408 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.469604 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.469605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.469605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.469700 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.469748 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.469947 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.470007 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:41:14.470220 15608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.460551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.3:43955)
0619 02:41:14.460685 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.460766 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.461270 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.461547 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.461550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.461899 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.462150 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.462155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.462840 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.462841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.463639 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.463641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.464168 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.464169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.464283 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.464781 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.464792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.467369 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.469082 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.469088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.469089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.469290 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.469463 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.469464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.469464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.469551 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.469751 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.469755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.469856 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.469937 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.470045 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260619 02:41:14.470634 15715 heartbeater.cc:347] Connected to a master server at 127.7.137.62:39009
I20260619 02:41:14.470717 15715 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:14.470893 15715 heartbeater.cc:511] Master 127.7.137.62:39009 requested a full tablet report, sending...
I20260619 02:41:14.471271 15232 ts_manager.cc:194] Registered new tserver with Master: b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:14.471755 15232 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:43955
I20260619 02:41:14.475796 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15584
I20260619 02:41:14.475941 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:14.476207 7716 external_mini_cluster.cc:1428] Setting key 41b6ca86ef617ec31fce2279a824a982
I20260619 02:41:14.476804 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.471638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:56115 (local address 127.7.137.62:39009)
0619 02:41:14.471676 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.471678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.471737 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:41:14.471803 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.471803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.471830 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.471885 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.472334 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.472872 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.473560 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.473763 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.473836 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.474567 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.474568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.474568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.474575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.475619 (+ 1044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.475966 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.475967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.475967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.476019 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.476234 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.476235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.476235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.476355 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.476428 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.476618 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.476693 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:14.476811 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.471554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.7.137.3:56115)
0619 02:41:14.471681 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.471691 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.471749 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.471994 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.471995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.472191 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.472245 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.472247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.472959 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.472960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.473473 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.473475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.473834 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.473835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.473897 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.474231 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.474235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.474476 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.475707 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.475708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.475709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.475911 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.476080 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.476081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.476082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.476159 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.476433 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.476434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.476478 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.476574 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.476673 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":84,"threads_started":1}
I20260619 02:41:14.477267 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:41:14.497918 15282 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:14.492298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.7.137.62:39009)
0619 02:41:14.492361 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.492363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.492370 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:14.492390 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.492391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.492399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.492459 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.493113 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.493597 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.494472 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.494662 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.494737 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.497826 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50066: BlockingRecv error: recv got EOF from 127.0.0.1:50066 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.504791 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.498666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.7.137.62:39009)
0619 02:41:14.498725 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.498727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.498808 (+ 81us) server_negotiation.cc:407] Connection header received
0619 02:41:14.498873 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.498874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.498881 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.498928 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.499319 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.499781 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.500436 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.500605 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.500651 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.502714 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.502715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.502715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.502721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.503844 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.504132 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.504132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.504133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.504178 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.504329 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.504330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.504330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.504419 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.504489 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.504619 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.504673 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:41:14.506465 15232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:14.507973 15232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:14.514150 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.7.137.3:41929)
0619 02:41:14.510615 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.510620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.510693 (+ 73us) server_negotiation.cc:407] Connection header received
0619 02:41:14.510796 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.510799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.510815 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.510930 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.511116 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.512131 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513197 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.513736 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513842 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.513924 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.514019 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.514059 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":77,"threads_started":1}
I20260619 02:41:14.514142 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:41929 (local address 127.0.0.1:43236)
0619 02:41:14.510626 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.510634 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.510701 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.511004 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.511005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.511007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.511043 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.511044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.512221 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.512221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.513116 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.513119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.513842 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.513918 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.513933 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.514005 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.514061 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260619 02:41:14.514297 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45821 (local address 127.0.0.1:42662)
0619 02:41:14.510408 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.510454 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.510520 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.510867 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.510868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.510871 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.510926 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.510928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.511816 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.511817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.513013 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.513015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.514032 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.514033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.514099 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.514104 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.514151 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.514198 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":43,"threads_started":1}
I20260619 02:41:14.514554 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.7.137.2:45821)
0619 02:41:14.510663 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.510667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.510676 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:14.510712 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.510714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.510728 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.510800 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.511009 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.511727 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513097 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.513854 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513927 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.513995 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.514416 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.514459 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":87,"threads_started":1}
I20260619 02:41:14.514729 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.0.0.1:40550)
0619 02:41:14.510456 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.510466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.510542 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.510781 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.510783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.510786 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.510872 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.510874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.512182 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.512182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.513247 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.513249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.514529 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.514530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.514594 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.514598 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.514647 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.514689 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":77,"threads_started":1}
I20260619 02:41:14.514976 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.510397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.7.137.1:40975)
0619 02:41:14.510593 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.510599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.510610 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:41:14.510653 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.510655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.510671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.510762 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.510958 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.512092 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.513324 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.514060 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.514120 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.514810 (+ 690us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.514842 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.514898 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260619 02:41:14.514866 15649 tablet_service.cc:1467] Processing CreateTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (DEFAULT_TABLE table=test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.515213 15649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.515128 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (DEFAULT_TABLE table=test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.515417 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.517103 15369 tablet_service.cc:1467] Processing CreateTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (DEFAULT_TABLE table=test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.517346 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.518908 15743 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Bootstrap starting.
I20260619 02:41:14.519557 15744 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Bootstrap starting.
I20260619 02:41:14.519590 15743 tablet_bootstrap.cc:654] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.519914 15743 log.cc:824] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:14.520135 15744 tablet_bootstrap.cc:654] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.520452 15744 log.cc:824] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:14.520817 15746 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Bootstrap starting.
I20260619 02:41:14.520998 15743 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: No bootstrap required, opened a new log
I20260619 02:41:14.521073 15743 ts_tablet_manager.cc:1397] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:14.521183 15744 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: No bootstrap required, opened a new log
I20260619 02:41:14.521274 15744 ts_tablet_manager.cc:1397] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:14.521413 15746 tablet_bootstrap.cc:654] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.521735 15746 log.cc:824] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:14.522505 15746 tablet_bootstrap.cc:492] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: No bootstrap required, opened a new log
I20260619 02:41:14.522573 15746 ts_tablet_manager.cc:1397] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:14.523106 15744 raft_consensus.cc:348] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.523265 15744 raft_consensus.cc:374] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.523310 15744 raft_consensus.cc:729] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.523221 15743 raft_consensus.cc:348] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.523375 15743 raft_consensus.cc:374] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.523411 15743 raft_consensus.cc:729] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.523418 15744 consensus_queue.cc:260] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.523518 15743 consensus_queue.cc:260] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.523677 15744 ts_tablet_manager.cc:1428] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:41:14.523772 15743 ts_tablet_manager.cc:1428] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:41:14.523833 15575 heartbeater.cc:503] Master 127.7.137.62:39009 was elected leader, sending a full tablet report...
I20260619 02:41:14.523913 15715 heartbeater.cc:503] Master 127.7.137.62:39009 was elected leader, sending a full tablet report...
I20260619 02:41:14.524570 15746 raft_consensus.cc:348] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.524744 15746 raft_consensus.cc:374] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.524796 15746 raft_consensus.cc:729] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.524906 15746 consensus_queue.cc:260] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.525166 15746 ts_tablet_manager.cc:1428] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 02:41:14.525210 15435 heartbeater.cc:503] Master 127.7.137.62:39009 was elected leader, sending a full tablet report...
W20260619 02:41:14.569962 15576 tablet.cc:2369] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:41:14.663381 15436 tablet.cc:2369] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:14.674571 15749 raft_consensus.cc:482] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:14.674695 15749 raft_consensus.cc:504] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.675035 15749 leader_election.cc:288] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929), fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975)
I20260619 02:41:14.678043 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.675336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:52905 (local address 127.7.137.1:40975)
0619 02:41:14.675376 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.675378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.675392 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:41:14.675444 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.675445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.675455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.675543 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.675743 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.676267 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677276 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.677672 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677725 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.677734 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.677932 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.677967 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:41:14.678038 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.675245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.7.137.2:52905)
0619 02:41:14.675298 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.675319 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.675371 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.675587 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.675588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.675590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.675668 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.675671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.676344 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.676346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.677195 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.677197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.677757 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.677855 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.677860 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.677921 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.677972 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:41:14.678233 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.675322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:58453 (local address 127.7.137.3:41929)
0619 02:41:14.675376 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.675378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.675565 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:41:14.675595 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.675596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.675608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.675682 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.675865 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.676327 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677391 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.677801 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677915 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.677926 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.678092 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.678154 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:41:14.678412 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.675247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:41929 (local address 127.7.137.2:58453)
0619 02:41:14.675470 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.675481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.675540 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.675689 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.675689 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.675691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.675788 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.675790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.676444 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.676446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.677289 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.677291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.677936 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.677937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.678002 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.678006 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.678066 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.678250 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":57,"thread_start_us":83,"threads_started":1}
I20260619 02:41:14.678369 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a31c49f8ee405aa8bd8cba5fc0b4e5" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" is_pre_election: true
I20260619 02:41:14.678536 15389 raft_consensus.cc:2454] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:14.678725 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a31c49f8ee405aa8bd8cba5fc0b4e5" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e" is_pre_election: true
I20260619 02:41:14.678838 15669 raft_consensus.cc:2454] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:14.679193 15462 leader_election.cc:302] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f12a7e570c7e489eaa6b0a1516b867c4, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:14.679337 15749 raft_consensus.cc:2781] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:14.679395 15749 raft_consensus.cc:482] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:14.679418 15749 raft_consensus.cc:3037] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.680081 15749 raft_consensus.cc:504] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.680218 15749 leader_election.cc:288] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Requested vote from peers b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929), fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975)
I20260619 02:41:14.680354 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a31c49f8ee405aa8bd8cba5fc0b4e5" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e"
I20260619 02:41:14.680372 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a31c49f8ee405aa8bd8cba5fc0b4e5" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
I20260619 02:41:14.680431 15669 raft_consensus.cc:3037] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.680473 15389 raft_consensus.cc:3037] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.681197 15389 raft_consensus.cc:2454] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:14.681206 15669 raft_consensus.cc:2454] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:14.681357 15462 leader_election.cc:302] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f12a7e570c7e489eaa6b0a1516b867c4, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:14.681438 15749 raft_consensus.cc:2781] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:14.681587 15749 raft_consensus.cc:686] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Becoming Leader. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Running, Role: LEADER
I20260619 02:41:14.681694 15749 consensus_queue.cc:237] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } }
I20260619 02:41:14.682516 15231 catalog_manager.cc:5495] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:14.697307 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.694854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.7.137.2:45821)
0619 02:41:14.694930 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.694933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.694955 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:41:14.695017 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.695018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.695044 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.695098 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:14.695206 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.695715 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.696480 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.696663 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.696711 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.696934 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.697037 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.697100 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.697185 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.697203 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:41:14.699216 15669 raft_consensus.cc:1261] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.699216 15389 raft_consensus.cc:1261] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.699553 15749 consensus_queue.cc:1035] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.699700 15753 consensus_queue.cc:1035] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.703253 15757 mvcc.cc:205] Tried to move back new op lower bound from 7298403838765113344 to 7298403838696779776. Current Snapshot: MvccSnapshot[applied={T|T < 7298403838765113344}]
I20260619 02:41:14.704285 15231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:14.704396 15231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:14.705209 15231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Sending DeleteTablet for 3 replicas of tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5
I20260619 02:41:14.705545 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260619 02:41:14.705545 15649 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:43236
I20260619 02:41:14.705765 15369 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:40550
I20260619 02:41:14.705958 15766 tablet_replica.cc:330] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: stopping tablet replica
I20260619 02:41:14.706058 15766 raft_consensus.cc:2229] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:14.706008 15231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:14.706140 15231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:14.706152 15766 raft_consensus.cc:2258] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.706312 15765 tablet_replica.cc:330] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: stopping tablet replica
I20260619 02:41:14.706419 15765 raft_consensus.cc:2229] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:14.706537 15767 tablet_replica.cc:330] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: stopping tablet replica
I20260619 02:41:14.706591 15765 raft_consensus.cc:2258] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.707109 15766 ts_tablet_manager.cc:1905] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.707111 15765 ts_tablet_manager.cc:1905] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.707983 15509 tablet_service.cc:1467] Processing CreateTablet for tablet a5fd3b21277d42348907a91c677e0b9c (DEFAULT_TABLE table=test-table [id=e99011aa03e946bc9e1ef8f721619ddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.708230 15649 tablet_service.cc:1467] Processing CreateTablet for tablet a5fd3b21277d42348907a91c677e0b9c (DEFAULT_TABLE table=test-table [id=e99011aa03e946bc9e1ef8f721619ddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.708348 15649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fd3b21277d42348907a91c677e0b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.708868 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fd3b21277d42348907a91c677e0b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.709172 15765 ts_tablet_manager.cc:1918] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:14.709234 15765 log.cc:1196] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/40a31c49f8ee405aa8bd8cba5fc0b4e5
I20260619 02:41:14.709504 15765 ts_tablet_manager.cc:1939] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P b620b9073b8c490692cf6842cbe4836e: Deleting consensus metadata
I20260619 02:41:14.709861 15743 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Bootstrap starting.
I20260619 02:41:14.710095 15744 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Bootstrap starting.
I20260619 02:41:14.710265 15743 tablet_bootstrap.cc:654] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.710491 15744 tablet_bootstrap.cc:654] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.710760 15216 catalog_manager.cc:4841] TS b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929): tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (table test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]) successfully deleted
I20260619 02:41:14.707849 15369 tablet_service.cc:1467] Processing CreateTablet for tablet a5fd3b21277d42348907a91c677e0b9c (DEFAULT_TABLE table=test-table [id=e99011aa03e946bc9e1ef8f721619ddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.710980 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fd3b21277d42348907a91c677e0b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.711673 15766 ts_tablet_manager.cc:1918] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:14.711010 15767 raft_consensus.cc:2229] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:14.711907 15767 raft_consensus.cc:2258] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.712004 15746 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Bootstrap starting.
I20260619 02:41:14.712123 15766 log.cc:1196] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/40a31c49f8ee405aa8bd8cba5fc0b4e5
I20260619 02:41:14.712356 15743 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: No bootstrap required, opened a new log
I20260619 02:41:14.712387 15746 tablet_bootstrap.cc:654] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.712415 15743 ts_tablet_manager.cc:1397] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:41:14.712467 15766 ts_tablet_manager.cc:1939] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting consensus metadata
I20260619 02:41:14.712474 15767 ts_tablet_manager.cc:1905] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.712592 15743 raft_consensus.cc:348] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.712680 15743 raft_consensus.cc:374] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.712707 15743 raft_consensus.cc:729] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.712817 15743 consensus_queue.cc:260] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.712929 15743 ts_tablet_manager.cc:1428] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:14.713038 15219 catalog_manager.cc:4841] TS f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821): tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (table test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]) successfully deleted
I20260619 02:41:14.713318 15746 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: No bootstrap required, opened a new log
I20260619 02:41:14.713361 15746 ts_tablet_manager.cc:1397] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:14.713367 15744 tablet_bootstrap.cc:492] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: No bootstrap required, opened a new log
I20260619 02:41:14.713409 15744 ts_tablet_manager.cc:1397] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:41:14.713515 15746 raft_consensus.cc:348] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.713594 15746 raft_consensus.cc:374] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.713555 15744 raft_consensus.cc:348] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.713621 15746 raft_consensus.cc:729] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.713622 15744 raft_consensus.cc:374] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.713645 15744 raft_consensus.cc:729] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.713668 15746 consensus_queue.cc:260] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.713692 15744 consensus_queue.cc:260] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.713784 15746 ts_tablet_manager.cc:1428] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.713778 15744 ts_tablet_manager.cc:1428] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.717156 15767 ts_tablet_manager.cc:1918] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:14.717223 15767 log.cc:1196] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/40a31c49f8ee405aa8bd8cba5fc0b4e5
I20260619 02:41:14.717770 15767 ts_tablet_manager.cc:1939] T 40a31c49f8ee405aa8bd8cba5fc0b4e5 P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting consensus metadata
I20260619 02:41:14.718271 15218 catalog_manager.cc:4841] TS fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975): tablet 40a31c49f8ee405aa8bd8cba5fc0b4e5 (table test-table [id=47f16db0e1c94899a421ef3e27dfb8c6]) successfully deleted
W20260619 02:41:14.718369 15716 tablet.cc:2369] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:14.791577 15753 raft_consensus.cc:482] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:14.791683 15753 raft_consensus.cc:504] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.791833 15753 leader_election.cc:288] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:14.792052 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fd3b21277d42348907a91c677e0b9c" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" is_pre_election: true
I20260619 02:41:14.792052 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fd3b21277d42348907a91c677e0b9c" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e" is_pre_election: true
I20260619 02:41:14.792161 15669 raft_consensus.cc:2454] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:14.792161 15389 raft_consensus.cc:2454] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:14.792338 15460 leader_election.cc:302] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b620b9073b8c490692cf6842cbe4836e, f12a7e570c7e489eaa6b0a1516b867c4; no voters:
I20260619 02:41:14.792440 15753 raft_consensus.cc:2781] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:14.792495 15753 raft_consensus.cc:482] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:14.792514 15753 raft_consensus.cc:3037] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.793430 15753 raft_consensus.cc:504] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.793565 15753 leader_election.cc:288] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Requested vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:14.793777 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fd3b21277d42348907a91c677e0b9c" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e"
I20260619 02:41:14.793821 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fd3b21277d42348907a91c677e0b9c" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
I20260619 02:41:14.793913 15389 raft_consensus.cc:3037] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.793911 15669 raft_consensus.cc:3037] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.794615 15669 raft_consensus.cc:2454] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:14.794695 15389 raft_consensus.cc:2454] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:14.794806 15460 leader_election.cc:302] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b620b9073b8c490692cf6842cbe4836e, f12a7e570c7e489eaa6b0a1516b867c4; no voters:
I20260619 02:41:14.794904 15753 raft_consensus.cc:2781] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:14.794957 15753 raft_consensus.cc:686] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Becoming Leader. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Running, Role: LEADER
I20260619 02:41:14.795023 15753 consensus_queue.cc:237] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.795668 15232 catalog_manager.cc:5495] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836873, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.62@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.828305 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.821480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:39009 (local address 127.0.0.1:50084)
0619 02:41:14.821624 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.821635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.821701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.821875 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.821876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.822156 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:14.822202 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.822204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.823072 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.823073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.823699 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.823701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.824062 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.824063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.824104 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.824652 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:14.824657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:14.826015 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:14.827304 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.827309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.827311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.827503 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.827664 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:14.827664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:14.827664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:14.827717 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:14.828045 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:14.828050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:14.828088 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.828143 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.828227 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260619 02:41:14.828367 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.821555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.7.137.62:39009)
0619 02:41:14.821611 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.821613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.821709 (+ 96us) server_negotiation.cc:407] Connection header received
0619 02:41:14.821741 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.821742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.821753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.821827 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.822298 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.822992 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.823785 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.823981 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.824038 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.826091 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.826092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.826092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.826098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.827236 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.827564 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.827565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.827565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.827610 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.827865 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.827866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.827866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.827974 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.828051 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.828176 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.828275 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":31}
I20260619 02:41:14.830338 15232 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:50084:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:41:14.830508 15232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:14.832031 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb153896ab243a1bee4f7f469242688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67bb399b60e6403b8a347a8e1fdd2fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:41:14.832050 15369 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb153896ab243a1bee4f7f469242688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67bb399b60e6403b8a347a8e1fdd2fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:41:14.832188 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb153896ab243a1bee4f7f469242688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.832201 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb153896ab243a1bee4f7f469242688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.833555 15744 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: Bootstrap starting.
I20260619 02:41:14.833776 15649 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb153896ab243a1bee4f7f469242688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67bb399b60e6403b8a347a8e1fdd2fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:41:14.833897 15649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb153896ab243a1bee4f7f469242688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.833927 15744 tablet_bootstrap.cc:654] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.834450 15744 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: No bootstrap required, opened a new log
I20260619 02:41:14.834510 15744 ts_tablet_manager.cc:1397] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:14.834658 15744 raft_consensus.cc:348] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.834739 15744 raft_consensus.cc:374] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.834771 15744 raft_consensus.cc:729] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.834816 15744 consensus_queue.cc:260] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.834911 15744 ts_tablet_manager.cc:1428] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.834942 15753 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.835049 15753 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.835234 15746 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: Bootstrap starting.
I20260619 02:41:14.835671 15746 tablet_bootstrap.cc:654] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.835966 15743 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: Bootstrap starting.
I20260619 02:41:14.836237 15746 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: No bootstrap required, opened a new log
I20260619 02:41:14.836272 15746 ts_tablet_manager.cc:1397] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:14.836328 15743 tablet_bootstrap.cc:654] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.836373 15746 raft_consensus.cc:348] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.836416 15746 raft_consensus.cc:374] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.836429 15746 raft_consensus.cc:729] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.836464 15746 consensus_queue.cc:260] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.836581 15746 ts_tablet_manager.cc:1428] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.836635 15751 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.836737 15751 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.837109 15743 tablet_bootstrap.cc:492] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: No bootstrap required, opened a new log
I20260619 02:41:14.837158 15743 ts_tablet_manager.cc:1397] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:14.837330 15743 raft_consensus.cc:348] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.837407 15743 raft_consensus.cc:374] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.837431 15743 raft_consensus.cc:729] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.837481 15743 consensus_queue.cc:260] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.837606 15743 ts_tablet_manager.cc:1428] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.837673 15750 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.837782 15750 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.850832 15751 raft_consensus.cc:482] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:14.850939 15751 raft_consensus.cc:504] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.851270 15751 leader_election.cc:288] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:14.854496 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.851506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:43793 (local address 127.7.137.2:45821)
0619 02:41:14.851554 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.851556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.851699 (+ 143us) server_negotiation.cc:407] Connection header received
0619 02:41:14.851762 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.851763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.851777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.851845 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.852049 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.852477 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.853613 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.854159 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.854230 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.854241 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.854387 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.854427 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:41:14.854513 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.851450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45821 (local address 127.7.137.1:43793)
0619 02:41:14.851635 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.851647 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.851705 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.851890 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.851891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.851893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.851962 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.851964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.852571 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.852572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.853534 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.853536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.854234 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.854235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.854314 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.854318 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.854362 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.854410 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260619 02:41:14.854524 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.851615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:43061 (local address 127.7.137.3:41929)
0619 02:41:14.851697 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.851699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.851726 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:41:14.851751 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.851752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.851761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.851812 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:14.851982 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.852468 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.853656 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.854165 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.854246 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.854255 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.854426 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.854462 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:41:14.854712 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb153896ab243a1bee4f7f469242688" candidate_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" is_pre_election: true
I20260619 02:41:14.854862 15529 raft_consensus.cc:2454] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fea4bf9ab3ec46d199c32e8900bb4c8c in term 0.
I20260619 02:41:14.854972 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.851450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:41929 (local address 127.7.137.1:43061)
0619 02:41:14.851622 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.851634 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.851692 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.851834 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.851835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.851837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:14.851912 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.851914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.852553 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.852554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.853548 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.853551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.854259 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.854260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.854336 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.854339 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.854413 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.854450 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260619 02:41:14.855119 15323 leader_election.cc:302] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f12a7e570c7e489eaa6b0a1516b867c4, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:14.855257 15751 raft_consensus.cc:2781] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:14.855319 15751 raft_consensus.cc:482] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:14.855348 15751 raft_consensus.cc:3037] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.855517 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb153896ab243a1bee4f7f469242688" candidate_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e" is_pre_election: true
I20260619 02:41:14.855626 15669 raft_consensus.cc:2454] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fea4bf9ab3ec46d199c32e8900bb4c8c in term 0.
I20260619 02:41:14.856261 15751 raft_consensus.cc:504] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.856392 15751 leader_election.cc:288] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [CANDIDATE]: Term 1 election: Requested vote from peers f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:14.856534 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb153896ab243a1bee4f7f469242688" candidate_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e"
I20260619 02:41:14.856601 15669 raft_consensus.cc:3037] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.857177 15669 raft_consensus.cc:2454] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fea4bf9ab3ec46d199c32e8900bb4c8c in term 1.
I20260619 02:41:14.857295 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb153896ab243a1bee4f7f469242688" candidate_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f12a7e570c7e489eaa6b0a1516b867c4"
I20260619 02:41:14.857339 15320 leader_election.cc:302] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b620b9073b8c490692cf6842cbe4836e, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:14.857373 15529 raft_consensus.cc:3037] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:14.857451 15751 raft_consensus.cc:2781] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:14.857596 15751 raft_consensus.cc:686] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 LEADER]: Becoming Leader. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Running, Role: LEADER
I20260619 02:41:14.857666 15751 consensus_queue.cc:237] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.857973 15783 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: TxnStatusTablet state changed. Reason: New leader fea4bf9ab3ec46d199c32e8900bb4c8c. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.858071 15783 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:41:14.858228 15529 raft_consensus.cc:2454] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fea4bf9ab3ec46d199c32e8900bb4c8c in term 1.
I20260619 02:41:14.858304 15785 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:41:14.858424 15785 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:41:14.858333 15232 catalog_manager.cc:5495] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1). New cstate: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:14.865327 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.862308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.0.0.1:40554)
0619 02:41:14.862418 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.862429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.862475 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.862642 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.862643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.862643 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:41:14.862698 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.862700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.863580 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.863580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.864212 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.864215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.864677 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.864678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.864787 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.864802 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.865139 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.865152 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.865217 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.865258 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 02:41:14.865326 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.862351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.7.137.1:40975)
0619 02:41:14.862412 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.862414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.862461 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:41:14.862539 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.862540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.862559 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.862635 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:14.862758 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.863497 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.864290 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.864600 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.864667 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.864895 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.865009 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.865094 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.865231 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.865254 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:41:14.866559 15669 raft_consensus.cc:1261] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Refusing update from remote peer fea4bf9ab3ec46d199c32e8900bb4c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.866631 15529 raft_consensus.cc:1261] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Refusing update from remote peer fea4bf9ab3ec46d199c32e8900bb4c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.866799 15783 consensus_queue.cc:1035] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.866904 15751 consensus_queue.cc:1035] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.867512 15753 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: TxnStatusTablet state changed. Reason: New leader fea4bf9ab3ec46d199c32e8900bb4c8c. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.867601 15753 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.867599 15750 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: TxnStatusTablet state changed. Reason: New leader fea4bf9ab3ec46d199c32e8900bb4c8c. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.867669 15750 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.868449 15751 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.868492 15750 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.868541 15751 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:41:14.868572 15750 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P b620b9073b8c490692cf6842cbe4836e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.868594 15751 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.868646 15751 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P fea4bf9ab3ec46d199c32e8900bb4c8c: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:41:14.869608 15756 mvcc.cc:205] Tried to move back new op lower bound from 7298403839451398144 to 7298403839417475072. Current Snapshot: MvccSnapshot[applied={T|T < 7298403839451398144}]
I20260619 02:41:14.869681 15753 tablet_replica.cc:439] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } } }
I20260619 02:41:14.869746 15753 tablet_replica.cc:442] T 0cb153896ab243a1bee4f7f469242688 P f12a7e570c7e489eaa6b0a1516b867c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:41:14.875311 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.872236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:38995 (local address 127.7.137.1:40975)
0619 02:41:14.872273 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.872275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.872961 (+ 686us) server_negotiation.cc:407] Connection header received
0619 02:41:14.873016 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.873017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.873026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.873078 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:14.873183 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.873610 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.874707 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.874888 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.874942 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.875116 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.875139 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.875180 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.875250 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.875264 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:14.875370 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.872146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.7.137.2:38995)
0619 02:41:14.872883 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.872895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.872959 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.873082 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.873083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.873084 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:41:14.873130 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.873132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.873678 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.873679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.874624 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.874627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.874960 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.874961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.875022 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.875037 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.875195 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.875197 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.875248 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.875293 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":71,"threads_started":1}
I20260619 02:41:14.877187 15669 raft_consensus.cc:1261] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.877384 15389 raft_consensus.cc:1261] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:14.877430 15749 consensus_queue.cc:1035] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.877590 15749 consensus_queue.cc:1035] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:14.885591 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.882587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45821 (local address 127.7.137.1:56161)
0619 02:41:14.882747 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:14.882757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:14.882803 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:14.882915 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:14.882916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:14.882917 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:41:14.882960 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.882961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.883803 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.883804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.884695 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:14.884697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.885224 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.885225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:14.885272 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.885285 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.885462 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.885463 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:41:14.885493 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 02:41:14.885522 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260619 02:41:14.885591 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.882626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:56161 (local address 127.7.137.2:45821)
0619 02:41:14.882662 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.882664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.882814 (+ 150us) server_negotiation.cc:407] Connection header received
0619 02:41:14.882838 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.882839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.882849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.882915 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:14.882999 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.883724 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.884820 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.885137 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.885210 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.885372 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:14.885407 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:14.885464 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.885496 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.885516 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:14.893594 15231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:14.893697 15231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:14.894285 15231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Sending DeleteTablet for 3 replicas of tablet a5fd3b21277d42348907a91c677e0b9c
I20260619 02:41:14.894517 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fd3b21277d42348907a91c677e0b9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260619 02:41:14.894527 15369 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fd3b21277d42348907a91c677e0b9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:40550
I20260619 02:41:14.894605 15767 tablet_replica.cc:330] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: stopping tablet replica
I20260619 02:41:14.894591 15649 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fd3b21277d42348907a91c677e0b9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:14 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:43236
I20260619 02:41:14.894645 15767 raft_consensus.cc:2229] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:14.894618 15766 tablet_replica.cc:330] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: stopping tablet replica
I20260619 02:41:14.894678 15765 tablet_replica.cc:330] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: stopping tablet replica
I20260619 02:41:14.894682 15767 raft_consensus.cc:2258] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.894701 15766 raft_consensus.cc:2229] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:14.894718 15765 raft_consensus.cc:2229] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:14.894749 15765 raft_consensus.cc:2258] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.894755 15766 raft_consensus.cc:2258] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:14.895123 15766 ts_tablet_manager.cc:1905] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.895290 15765 ts_tablet_manager.cc:1905] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.896603 15767 ts_tablet_manager.cc:1905] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:14.896706 15765 ts_tablet_manager.cc:1918] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:41:14.896802 15765 log.cc:1196] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/a5fd3b21277d42348907a91c677e0b9c
I20260619 02:41:14.896970 15766 ts_tablet_manager.cc:1918] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:41:14.897027 15766 log.cc:1196] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/a5fd3b21277d42348907a91c677e0b9c
I20260619 02:41:14.897035 15765 ts_tablet_manager.cc:1939] T a5fd3b21277d42348907a91c677e0b9c P b620b9073b8c490692cf6842cbe4836e: Deleting consensus metadata
I20260619 02:41:14.897261 15766 ts_tablet_manager.cc:1939] T a5fd3b21277d42348907a91c677e0b9c P f12a7e570c7e489eaa6b0a1516b867c4: Deleting consensus metadata
I20260619 02:41:14.897468 15216 catalog_manager.cc:4841] TS b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929): tablet a5fd3b21277d42348907a91c677e0b9c (table test-table [id=e99011aa03e946bc9e1ef8f721619ddb]) successfully deleted
I20260619 02:41:14.897913 15219 catalog_manager.cc:4841] TS f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821): tablet a5fd3b21277d42348907a91c677e0b9c (table test-table [id=e99011aa03e946bc9e1ef8f721619ddb]) successfully deleted
I20260619 02:41:14.898217 15767 ts_tablet_manager.cc:1918] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:41:14.898270 15767 log.cc:1196] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/a5fd3b21277d42348907a91c677e0b9c
I20260619 02:41:14.898464 15767 ts_tablet_manager.cc:1939] T a5fd3b21277d42348907a91c677e0b9c P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting consensus metadata
I20260619 02:41:14.898852 15218 catalog_manager.cc:4841] TS fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975): tablet a5fd3b21277d42348907a91c677e0b9c (table test-table [id=e99011aa03e946bc9e1ef8f721619ddb]) successfully deleted
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:41:14.921173 15282 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:14.915133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.7.137.62:39009)
0619 02:41:14.915205 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.915207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.915306 (+ 99us) server_negotiation.cc:407] Connection header received
0619 02:41:14.915399 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.915400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.915412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.915468 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.916019 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.916869 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.917653 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.917959 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.918032 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.921106 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50088: BlockingRecv error: recv got EOF from 127.0.0.1:50088 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:41:14 dist-test-slave-pdnb krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM
I20260619 02:41:14.929138 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:14.921968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.7.137.62:39009)
0619 02:41:14.922029 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:14.922031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:14.922094 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:41:14.922125 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:14.922126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:14.922136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:14.922197 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:14.922620 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.923322 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.924025 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:14.924320 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:14.924398 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:14.926449 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:14.926450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:14.926450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:14.926457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:14.927966 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.928276 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.928277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.928277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.928336 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:14.928501 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:14.928502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:14.928502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:14.928625 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:14.928750 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:14.928968 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 02:41:14.929043 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:41:14.931015 15231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:14.931191 15231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:14.932569 15369 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf54c2015774324956421eb24f3fa2b (DEFAULT_TABLE table=test-table [id=f222a4df7bc641b5a610d68ea2f679c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.932579 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf54c2015774324956421eb24f3fa2b (DEFAULT_TABLE table=test-table [id=f222a4df7bc641b5a610d68ea2f679c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.932708 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf54c2015774324956421eb24f3fa2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.932782 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf54c2015774324956421eb24f3fa2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.934113 15744 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Bootstrap starting.
I20260619 02:41:14.934422 15744 tablet_bootstrap.cc:654] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.934679 15746 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Bootstrap starting.
I20260619 02:41:14.935005 15746 tablet_bootstrap.cc:654] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.935065 15744 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: No bootstrap required, opened a new log
I20260619 02:41:14.935112 15744 ts_tablet_manager.cc:1397] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:14.935230 15744 raft_consensus.cc:348] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.935328 15744 raft_consensus.cc:374] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.935314 15649 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf54c2015774324956421eb24f3fa2b (DEFAULT_TABLE table=test-table [id=f222a4df7bc641b5a610d68ea2f679c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:14.935359 15744 raft_consensus.cc:729] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.935400 15744 consensus_queue.cc:260] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.935431 15649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf54c2015774324956421eb24f3fa2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:14.935496 15744 ts_tablet_manager.cc:1428] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:14.935838 15746 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: No bootstrap required, opened a new log
I20260619 02:41:14.935901 15746 ts_tablet_manager.cc:1397] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:14.936048 15746 raft_consensus.cc:348] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.936120 15746 raft_consensus.cc:374] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.936147 15746 raft_consensus.cc:729] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.936192 15746 consensus_queue.cc:260] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.936287 15746 ts_tablet_manager.cc:1428] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:14.936642 15743 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Bootstrap starting.
I20260619 02:41:14.937217 15743 tablet_bootstrap.cc:654] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:14.937870 15743 tablet_bootstrap.cc:492] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: No bootstrap required, opened a new log
I20260619 02:41:14.937922 15743 ts_tablet_manager.cc:1397] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:14.938059 15743 raft_consensus.cc:348] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.938134 15743 raft_consensus.cc:374] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:14.938153 15743 raft_consensus.cc:729] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Initialized, Role: FOLLOWER
I20260619 02:41:14.938192 15743 consensus_queue.cc:260] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:14.938282 15743 ts_tablet_manager.cc:1428] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:15.030720 15750 raft_consensus.cc:482] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:15.030819 15750 raft_consensus.cc:504] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.031116 15750 leader_election.cc:288] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821)
I20260619 02:41:15.035144 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.031371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:56697 (local address 127.7.137.1:40975)
0619 02:41:15.031431 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:15.031434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:15.031497 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:41:15.031592 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:15.031593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:15.031604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:15.031662 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:15.031870 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.032624 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034236 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.034780 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034856 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.034867 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:15.035039 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 02:41:15.035072 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:41:15.035144 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.031369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:36517 (local address 127.7.137.2:45821)
0619 02:41:15.031419 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:15.031421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:15.031545 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:41:15.031592 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:15.031593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:15.031607 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:15.031661 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:15.031870 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.032632 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034236 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.034780 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034856 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.034867 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:15.035031 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:41:15.035072 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:41:15.035168 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.031290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45821 (local address 127.7.137.3:36517)
0619 02:41:15.031467 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:15.031476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:15.031526 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:15.031679 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:15.031680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:15.031684 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:15.031790 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:15.031791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.032712 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.032713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:15.034147 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:15.034150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.034871 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:15.034942 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.034948 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:41:15.035009 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:41:15.035056 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":123,"thread_start_us":58,"threads_started":1}
I20260619 02:41:15.035171 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.031290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.7.137.3:56697)
0619 02:41:15.031419 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:15.031431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:15.031502 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:15.031679 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:15.031680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:15.031683 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:15.031789 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:15.031791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.032740 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.032741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:15.034147 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:15.034150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.034870 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.034871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:15.034960 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.034963 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:15.035016 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:41:15.035062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":61,"thread_start_us":58,"threads_started":1}
I20260619 02:41:15.035583 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf54c2015774324956421eb24f3fa2b" candidate_uuid: "b620b9073b8c490692cf6842cbe4836e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" is_pre_election: true
I20260619 02:41:15.035621 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf54c2015774324956421eb24f3fa2b" candidate_uuid: "b620b9073b8c490692cf6842cbe4836e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" is_pre_election: true
I20260619 02:41:15.035705 15529 raft_consensus.cc:2454] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b620b9073b8c490692cf6842cbe4836e in term 0.
I20260619 02:41:15.035703 15389 raft_consensus.cc:2454] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b620b9073b8c490692cf6842cbe4836e in term 0.
I20260619 02:41:15.035871 15602 leader_election.cc:302] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b620b9073b8c490692cf6842cbe4836e, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:15.036047 15750 raft_consensus.cc:2781] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:15.036098 15750 raft_consensus.cc:482] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:15.036139 15750 raft_consensus.cc:3037] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.036865 15750 raft_consensus.cc:504] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.037007 15750 leader_election.cc:288] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [CANDIDATE]: Term 1 election: Requested vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821)
I20260619 02:41:15.037151 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf54c2015774324956421eb24f3fa2b" candidate_uuid: "b620b9073b8c490692cf6842cbe4836e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f12a7e570c7e489eaa6b0a1516b867c4"
I20260619 02:41:15.037169 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf54c2015774324956421eb24f3fa2b" candidate_uuid: "b620b9073b8c490692cf6842cbe4836e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
I20260619 02:41:15.037228 15529 raft_consensus.cc:3037] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.037228 15389 raft_consensus.cc:3037] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.037947 15529 raft_consensus.cc:2454] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b620b9073b8c490692cf6842cbe4836e in term 1.
I20260619 02:41:15.037947 15389 raft_consensus.cc:2454] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b620b9073b8c490692cf6842cbe4836e in term 1.
I20260619 02:41:15.038112 15602 leader_election.cc:302] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b620b9073b8c490692cf6842cbe4836e, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:15.038215 15750 raft_consensus.cc:2781] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:15.038311 15750 raft_consensus.cc:686] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 1 LEADER]: Becoming Leader. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Running, Role: LEADER
I20260619 02:41:15.038417 15750 consensus_queue.cc:237] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.039095 15231 catalog_manager.cc:5495] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e reported cstate change: term changed from 0 to 1, leader changed from <none> to b620b9073b8c490692cf6842cbe4836e (127.7.137.3). New cstate: current_term: 1 leader_uuid: "b620b9073b8c490692cf6842cbe4836e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } health_report { overall_health: HEALTHY } } }
I20260619 02:41:15.055976 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.053622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.7.137.3:41929)
0619 02:41:15.053701 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:15.053703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:15.053712 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:15.053749 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:15.053751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:15.053772 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:15.053840 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:15.053998 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.054460 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.055198 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.055409 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.055484 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.055643 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:15.055754 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:15.055835 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:15.055873 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:41:15.055891 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:41:15.056888 15389 raft_consensus.cc:1261] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Refusing update from remote peer b620b9073b8c490692cf6842cbe4836e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:15.056918 15529 raft_consensus.cc:1261] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Refusing update from remote peer b620b9073b8c490692cf6842cbe4836e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:15.057175 15750 consensus_queue.cc:1035] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:15.057296 15817 consensus_queue.cc:1035] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:15.060395 15232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:15.060483 15232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:15.061141 15232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Sending DeleteTablet for 3 replicas of tablet 3bf54c2015774324956421eb24f3fa2b
I20260619 02:41:15.061425 15369 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf54c2015774324956421eb24f3fa2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:40550
I20260619 02:41:15.061503 15649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf54c2015774324956421eb24f3fa2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:43236
I20260619 02:41:15.061518 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf54c2015774324956421eb24f3fa2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260619 02:41:15.061578 15765 tablet_replica.cc:330] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: stopping tablet replica
I20260619 02:41:15.061595 15767 tablet_replica.cc:330] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: stopping tablet replica
I20260619 02:41:15.061618 15765 raft_consensus.cc:2229] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:15.061642 15767 raft_consensus.cc:2229] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:15.061678 15767 raft_consensus.cc:2258] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:15.061695 15765 raft_consensus.cc:2258] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:15.061738 15766 tablet_replica.cc:330] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: stopping tablet replica
I20260619 02:41:15.061722 15232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:41:15.061789 15766 raft_consensus.cc:2229] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:15.061830 15766 raft_consensus.cc:2258] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:41:15.061848 15232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:15.062346 15767 ts_tablet_manager.cc:1905] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:15.062925 15369 tablet_service.cc:1467] Processing CreateTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 (DEFAULT_TABLE table=test-table [id=6e338b01eeaa4f7a98315ebb0fcabff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:15.063035 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 955fb3e6f6df471cbb08f9e245c4dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:15.063244 15766 ts_tablet_manager.cc:1905] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:15.063808 15509 tablet_service.cc:1467] Processing CreateTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 (DEFAULT_TABLE table=test-table [id=6e338b01eeaa4f7a98315ebb0fcabff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:15.063903 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 955fb3e6f6df471cbb08f9e245c4dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:15.064334 15649 tablet_service.cc:1467] Processing CreateTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 (DEFAULT_TABLE table=test-table [id=6e338b01eeaa4f7a98315ebb0fcabff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:15.064425 15649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 955fb3e6f6df471cbb08f9e245c4dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:15.064810 15766 ts_tablet_manager.cc:1918] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:15.064858 15766 log.cc:1196] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/3bf54c2015774324956421eb24f3fa2b
I20260619 02:41:15.065080 15766 ts_tablet_manager.cc:1939] T 3bf54c2015774324956421eb24f3fa2b P f12a7e570c7e489eaa6b0a1516b867c4: Deleting consensus metadata
I20260619 02:41:15.065276 15744 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Bootstrap starting.
I20260619 02:41:15.065409 15219 catalog_manager.cc:4841] TS f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821): tablet 3bf54c2015774324956421eb24f3fa2b (table test-table [id=f222a4df7bc641b5a610d68ea2f679c1]) successfully deleted
I20260619 02:41:15.065549 15744 tablet_bootstrap.cc:654] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:15.065788 15743 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Bootstrap starting.
I20260619 02:41:15.065821 15746 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: Bootstrap starting.
I20260619 02:41:15.066126 15746 tablet_bootstrap.cc:654] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:15.066157 15743 tablet_bootstrap.cc:654] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:15.066632 15765 ts_tablet_manager.cc:1905] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:15.066710 15746 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: No bootstrap required, opened a new log
I20260619 02:41:15.066747 15746 ts_tablet_manager.cc:1397] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:15.066838 15746 raft_consensus.cc:348] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.066898 15746 raft_consensus.cc:374] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:15.066919 15746 raft_consensus.cc:729] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea4bf9ab3ec46d199c32e8900bb4c8c, State: Initialized, Role: FOLLOWER
I20260619 02:41:15.066969 15746 consensus_queue.cc:260] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.067049 15746 ts_tablet_manager.cc:1428] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:15.067945 15765 ts_tablet_manager.cc:1918] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:15.067994 15765 log.cc:1196] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/3bf54c2015774324956421eb24f3fa2b
I20260619 02:41:15.068195 15765 ts_tablet_manager.cc:1939] T 3bf54c2015774324956421eb24f3fa2b P b620b9073b8c490692cf6842cbe4836e: Deleting consensus metadata
I20260619 02:41:15.068207 15767 ts_tablet_manager.cc:1918] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:15.068250 15767 log.cc:1196] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/3bf54c2015774324956421eb24f3fa2b
I20260619 02:41:15.068444 15767 ts_tablet_manager.cc:1939] T 3bf54c2015774324956421eb24f3fa2b P fea4bf9ab3ec46d199c32e8900bb4c8c: Deleting consensus metadata
I20260619 02:41:15.068706 15218 catalog_manager.cc:4841] TS fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975): tablet 3bf54c2015774324956421eb24f3fa2b (table test-table [id=f222a4df7bc641b5a610d68ea2f679c1]) successfully deleted
I20260619 02:41:15.068821 15216 catalog_manager.cc:4841] TS b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929): tablet 3bf54c2015774324956421eb24f3fa2b (table test-table [id=f222a4df7bc641b5a610d68ea2f679c1]) successfully deleted
I20260619 02:41:15.069090 15743 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: No bootstrap required, opened a new log
I20260619 02:41:15.069130 15743 ts_tablet_manager.cc:1397] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 02:41:15.069271 15743 raft_consensus.cc:348] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.069335 15743 raft_consensus.cc:374] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:15.069350 15743 raft_consensus.cc:729] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b620b9073b8c490692cf6842cbe4836e, State: Initialized, Role: FOLLOWER
I20260619 02:41:15.069409 15743 consensus_queue.cc:260] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.069506 15743 ts_tablet_manager.cc:1428] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:15.076650 15744 tablet_bootstrap.cc:492] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: No bootstrap required, opened a new log
I20260619 02:41:15.076735 15744 ts_tablet_manager.cc:1397] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.000s
I20260619 02:41:15.076846 15744 raft_consensus.cc:348] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.076921 15744 raft_consensus.cc:374] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:15.076941 15744 raft_consensus.cc:729] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Initialized, Role: FOLLOWER
I20260619 02:41:15.076984 15744 consensus_queue.cc:260] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.077077 15744 ts_tablet_manager.cc:1428] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:15.197722 15749 raft_consensus.cc:482] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:15.197826 15749 raft_consensus.cc:504] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.197983 15749 leader_election.cc:288] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:15.198210 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "955fb3e6f6df471cbb08f9e245c4dcd2" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" is_pre_election: true
I20260619 02:41:15.198292 15389 raft_consensus.cc:2454] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:15.198225 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "955fb3e6f6df471cbb08f9e245c4dcd2" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e" is_pre_election: true
I20260619 02:41:15.198341 15669 raft_consensus.cc:2454] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 0.
I20260619 02:41:15.198463 15462 leader_election.cc:302] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f12a7e570c7e489eaa6b0a1516b867c4, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:15.198559 15749 raft_consensus.cc:2781] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:15.198599 15749 raft_consensus.cc:482] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:15.198623 15749 raft_consensus.cc:3037] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.199235 15749 raft_consensus.cc:504] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.199370 15749 leader_election.cc:288] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Requested vote from peers fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975), b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929)
I20260619 02:41:15.199532 15669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "955fb3e6f6df471cbb08f9e245c4dcd2" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b620b9073b8c490692cf6842cbe4836e"
I20260619 02:41:15.199512 15389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "955fb3e6f6df471cbb08f9e245c4dcd2" candidate_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c"
I20260619 02:41:15.199602 15669 raft_consensus.cc:3037] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.199604 15389 raft_consensus.cc:3037] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:15.200333 15669 raft_consensus.cc:2454] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:15.200333 15389 raft_consensus.cc:2454] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f12a7e570c7e489eaa6b0a1516b867c4 in term 1.
I20260619 02:41:15.200523 15462 leader_election.cc:302] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f12a7e570c7e489eaa6b0a1516b867c4, fea4bf9ab3ec46d199c32e8900bb4c8c; no voters:
I20260619 02:41:15.200626 15749 raft_consensus.cc:2781] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:15.200690 15749 raft_consensus.cc:686] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Becoming Leader. State: Replica: f12a7e570c7e489eaa6b0a1516b867c4, State: Running, Role: LEADER
I20260619 02:41:15.200778 15749 consensus_queue.cc:237] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } }
I20260619 02:41:15.201375 15231 catalog_manager.cc:5495] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f12a7e570c7e489eaa6b0a1516b867c4" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 } health_report { overall_health: UNKNOWN } } }
I20260619 02:41:15.213442 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.211491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.7.137.2:45821)
0619 02:41:15.211551 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:15.211553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:15.211561 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:15.211583 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:15.211584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:15.211596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:15.211644 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:15.211722 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.212175 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.212883 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:15.213060 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:15.213111 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:15.213218 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:15.213243 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:15.213275 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:15.213317 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:41:15.213333 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:41:15.215226 15389 raft_consensus.cc:1261] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:15.215284 15669 raft_consensus.cc:1261] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Refusing update from remote peer f12a7e570c7e489eaa6b0a1516b867c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:15.215464 15753 consensus_queue.cc:1035] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b620b9073b8c490692cf6842cbe4836e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:15.215586 15749 consensus_queue.cc:1035] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea4bf9ab3ec46d199c32e8900bb4c8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:15.216532 15758 mvcc.cc:205] Tried to move back new op lower bound from 7298403840880398336 to 7298403840822824960. Current Snapshot: MvccSnapshot[applied={T|T < 7298403840880398336}]
I20260619 02:41:15.224895 15231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:15.224977 15231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:15.225610 15231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab0d250f686e48cea3c795082092ba7b: Sending DeleteTablet for 3 replicas of tablet 955fb3e6f6df471cbb08f9e245c4dcd2
I20260619 02:41:15.225860 15509 tablet_service.cc:1514] Processing DeleteTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260619 02:41:15.225939 15649 tablet_service.cc:1514] Processing DeleteTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:43236
I20260619 02:41:15.225963 15766 tablet_replica.cc:330] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: stopping tablet replica
I20260619 02:41:15.226009 15766 raft_consensus.cc:2229] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:41:15.226019 15765 tablet_replica.cc:330] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: stopping tablet replica
I20260619 02:41:15.226054 15765 raft_consensus.cc:2229] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:15.226060 15766 raft_consensus.cc:2258] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:15.226081 15765 raft_consensus.cc:2258] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:15.226151 15369 tablet_service.cc:1514] Processing DeleteTablet for tablet 955fb3e6f6df471cbb08f9e245c4dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:15 UTC) from {username='oryx', principal='oryx/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:40550
I20260619 02:41:15.226194 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15304
I20260619 02:41:15.226327 15767 tablet_replica.cc:330] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c: stopping tablet replica
I20260619 02:41:15.226365 15766 ts_tablet_manager.cc:1905] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:15.226388 15767 raft_consensus.cc:2229] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:15.226439 15767 raft_consensus.cc:2258] T 955fb3e6f6df471cbb08f9e245c4dcd2 P fea4bf9ab3ec46d199c32e8900bb4c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:15.227367 15765 ts_tablet_manager.cc:1905] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:15.227795 15766 ts_tablet_manager.cc:1918] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:41:15.227844 15766 log.cc:1196] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/955fb3e6f6df471cbb08f9e245c4dcd2
I20260619 02:41:15.228066 15766 ts_tablet_manager.cc:1939] T 955fb3e6f6df471cbb08f9e245c4dcd2 P f12a7e570c7e489eaa6b0a1516b867c4: Deleting consensus metadata
I20260619 02:41:15.228377 15219 catalog_manager.cc:4841] TS f12a7e570c7e489eaa6b0a1516b867c4 (127.7.137.2:45821): tablet 955fb3e6f6df471cbb08f9e245c4dcd2 (table test-table [id=6e338b01eeaa4f7a98315ebb0fcabff3]) successfully deleted
I20260619 02:41:15.228910 15765 ts_tablet_manager.cc:1918] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:41:15.228946 15765 log.cc:1196] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/955fb3e6f6df471cbb08f9e245c4dcd2
I20260619 02:41:15.229097 15765 ts_tablet_manager.cc:1939] T 955fb3e6f6df471cbb08f9e245c4dcd2 P b620b9073b8c490692cf6842cbe4836e: Deleting consensus metadata
I20260619 02:41:15.229617 15216 catalog_manager.cc:4841] TS b620b9073b8c490692cf6842cbe4836e (127.7.137.3:41929): tablet 955fb3e6f6df471cbb08f9e245c4dcd2 (table test-table [id=6e338b01eeaa4f7a98315ebb0fcabff3]) successfully deleted
W20260619 02:41:15.233660 15218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:40975 (error 108)
I20260619 02:41:15.233902 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15444
I20260619 02:41:15.234665 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:15.234028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:40975 (local address 127.0.0.1:40558)
0619 02:41:15.234599 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:15.234633 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:40975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":40,"threads_started":1}
W20260619 02:41:15.234784 15218 catalog_manager.cc:4584] TS fea4bf9ab3ec46d199c32e8900bb4c8c (127.7.137.1:40975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 955fb3e6f6df471cbb08f9e245c4dcd2: Network error: Client connection negotiation failed: client connection to 127.7.137.1:40975: connect: Connection refused (error 111)
I20260619 02:41:15.240386 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15584
I20260619 02:41:15.245947 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15200
2026-06-19T02:41:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2542 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:15 dist-test-slave-pdnb krb5kdc[15835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:15 dist-test-slave-pdnb krb5kdc[15835](info): set up 2 sockets
Jun 19 02:41:15 dist-test-slave-pdnb krb5kdc[15835](info): commencing operation
krb5kdc: starting...
W20260619 02:41:16.273592 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:16 dist-test-slave-pdnb krb5kdc[15835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:16.387109 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42293
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:16.465692 15848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:16.465919 15848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:16.465953 15848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:16.467097 15848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:16.467160 15848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:16.467183 15848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:16.467203 15848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:16.467234 15848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:16.468741 15848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:42293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:16.469014 15848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:16.469233 15848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:16.473903 15856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:16.473945 15853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:16.474033 15848 server_base.cc:1034] running on GCE node
W20260619 02:41:16.473908 15854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:16.474339 15848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:16.474567 15848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:16.475701 15848 hybrid_clock.cc:648] HybridClock initialized: now 1781836876475665 us; error 54 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T02:41:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1238 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:16 dist-test-slave-pdnb krb5kdc[15861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:16 dist-test-slave-pdnb krb5kdc[15861](info): set up 2 sockets
Jun 19 02:41:16 dist-test-slave-pdnb krb5kdc[15861](info): commencing operation
krb5kdc: starting...
W20260619 02:41:17.517175 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:17 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:17.629716 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38887
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:38887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:17.708207 15874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:17.708420 15874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:17.708443 15874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:17.709669 15874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:17.709712 15874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:17.709726 15874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:17.709780 15874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:17.709798 15874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:17.711258 15874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:38887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:17.711512 15874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:17.711735 15874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:17.716480 15879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:17.716522 15882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:17.716480 15880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:17.716692 15874 server_base.cc:1034] running on GCE node
I20260619 02:41:17.716956 15874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:17.717213 15874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:17.718358 15874 hybrid_clock.cc:648] HybridClock initialized: now 1781836877718335 us; error 38 us; skew 500 ppm
Jun 19 02:41:17 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:17.721000 15874 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:17.721446 15874 webserver.cc:460] Webserver started at http://127.7.137.62:39227/ using document root <none> and password file <none>
I20260619 02:41:17.721698 15874 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:17.721771 15874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:17.721916 15874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:17.722997 15874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "66709428970c3408c2a089cd678d06ec"
server_key_iv: "8b3c97ab65a18816a71016792000241b"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.723371 15874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "66709428970c3408c2a089cd678d06ec"
server_key_iv: "8b3c97ab65a18816a71016792000241b"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.724855 15874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:17.725696 15889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.725893 15874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:17.725994 15874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "66709428970c3408c2a089cd678d06ec"
server_key_iv: "8b3c97ab65a18816a71016792000241b"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.726085 15874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:17.742096 15874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:17.744191 15874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:17.744361 15874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:17.748330 15874 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:38887
I20260619 02:41:17.748371 15941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:38887 every 8 connection(s)
I20260619 02:41:17.748775 15874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:17.749670 15942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:17.752101 15942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Bootstrap starting.
I20260619 02:41:17.752676 15942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:17.753013 15942 log.cc:824] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:17.753720 15942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: No bootstrap required, opened a new log
I20260619 02:41:17.754097 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15874
I20260619 02:41:17.754216 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:17.754415 7716 external_mini_cluster.cc:1428] Setting key 4c5abe02bd261e22e88aa3e74da72cc6
I20260619 02:41:17.756094 15942 raft_consensus.cc:348] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:17.756316 15942 raft_consensus.cc:374] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:17.756362 15942 raft_consensus.cc:729] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14496fc04fb4bfb8baf7eaf03d2c3f7, State: Initialized, Role: FOLLOWER
I20260619 02:41:17.756520 15942 consensus_queue.cc:260] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:17.756618 15942 raft_consensus.cc:388] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:17.756668 15942 raft_consensus.cc:482] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:17.756744 15942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:17.757344 15942 raft_consensus.cc:504] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:17.757432 15942 leader_election.cc:302] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b14496fc04fb4bfb8baf7eaf03d2c3f7; no voters:
I20260619 02:41:17.757654 15942 leader_election.cc:288] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:17.757683 15947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:17.757818 15947 raft_consensus.cc:686] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 LEADER]: Becoming Leader. State: Replica: b14496fc04fb4bfb8baf7eaf03d2c3f7, State: Running, Role: LEADER
I20260619 02:41:17.757931 15942 sys_catalog.cc:564] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:17.757926 15947 consensus_queue.cc:237] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:17.759793 15948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b14496fc04fb4bfb8baf7eaf03d2c3f7. Latest consensus state: current_term: 1 leader_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } } }
I20260619 02:41:17.759886 15948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:17.760061 15949 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } } }
I20260619 02:41:17.760142 15949 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: This master's current role is: LEADER
W20260619 02:41:17.760143 15962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:41:17.760249 15962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:41:17.760313 15964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:17.760658 15964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:17.762260 15964 catalog_manager.cc:1347] Generated new cluster ID: aa415961ab8745819a3e47b392f43361
I20260619 02:41:17.762321 15964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:41:17 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:17.771544 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:17.754959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.7.137.62:38887)
0619 02:41:17.755139 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:17.755146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:17.755171 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:41:17.755582 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:17.755589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:17.755635 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:17.755750 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:17.756647 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.758300 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.759501 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.760083 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.760209 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:17.767683 (+ 7474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:17.767688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:17.767688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:17.767702 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:17.769733 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.770189 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.770192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.770193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.770303 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.770531 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.770531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.770532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.770842 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:17.770992 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:17.771151 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:41:17.771292 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260619 02:41:17.777117 15964 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:17.777927 15964 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:17.785794 15964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Generated new TSK 0
I20260619 02:41:17.785990 15964 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:41:17.812070 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:17.893271 15968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:17.893496 15968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:17.893520 15968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:17.894711 15968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:17.894753 15968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:17.894780 15968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:17.896281 15968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:17.896524 15968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:17.896773 15968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:17.901695 15973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:17.901767 15976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:17.901701 15974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:17.901880 15968 server_base.cc:1034] running on GCE node
I20260619 02:41:17.902107 15968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:17.902348 15968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:17.903492 15968 hybrid_clock.cc:648] HybridClock initialized: now 1781836877903482 us; error 32 us; skew 500 ppm
Jun 19 02:41:17 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:17.906127 15968 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:41:17.906600 15968 webserver.cc:460] Webserver started at http://127.7.137.1:41549/ using document root <none> and password file <none>
I20260619 02:41:17.906826 15968 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:17.906870 15968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:17.907002 15968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:17.908046 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "506031e8d12444dda4c4d5f75bcfccad"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "07a62aeb0a0e52a3964ef842b746928c"
server_key_iv: "085e58e096b93a0b67607a89a3738ea3"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.908389 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "506031e8d12444dda4c4d5f75bcfccad"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "07a62aeb0a0e52a3964ef842b746928c"
server_key_iv: "085e58e096b93a0b67607a89a3738ea3"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.909731 15968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:41:17.910487 15983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.910653 15968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:17.910763 15968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "506031e8d12444dda4c4d5f75bcfccad"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "07a62aeb0a0e52a3964ef842b746928c"
server_key_iv: "085e58e096b93a0b67607a89a3738ea3"
server_key_version: "encryptionkey@0"
I20260619 02:41:17.910840 15968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:17.924017 15968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:17.926342 15968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:17.926507 15968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:17.926746 15968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:17.927059 15968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:17.927098 15968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.927120 15968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:17.927153 15968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:17.933455 15968 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:43953
I20260619 02:41:17.933501 16096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:43953 every 8 connection(s)
I20260619 02:41:17.933851 15968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:17.936853 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 15968
I20260619 02:41:17.936959 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:17.937165 7716 external_mini_cluster.cc:1428] Setting key 2d8c00c120247889bc64d2689d6cb8a6
Jun 19 02:41:17 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:17.944403 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:17.934634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:38175 (local address 127.7.137.62:38887)
0619 02:41:17.934713 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:17.934715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:17.934872 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:41:17.935330 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:17.935331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:17.935341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:17.935388 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:17.936265 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.936899 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.938197 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.938543 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.938615 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:17.941114 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:17.941115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:17.941116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:17.941124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:17.942905 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.943353 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.943355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.943355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.943423 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:17.943687 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:17.943688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:17.943689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:17.943850 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:17.943929 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:17.944189 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:41:17.944283 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:41:17.944480 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:17.934591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:38175)
0619 02:41:17.934751 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:17.934785 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:17.935251 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:17.935448 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:17.935451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:17.936003 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:17.936192 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:17.936197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.937031 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.937032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:17.938114 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:17.938117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:17.938629 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:17.938629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:17.938730 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:17.939317 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:17.939328 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:17.941018 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:17.943003 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:17.943010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:17.943012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:17.943270 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:17.943503 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:17.943503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:17.943504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:17.943611 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:17.943922 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:17.943927 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:17.944051 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:41:17.944139 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:41:17.944283 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260619 02:41:17.945032 16097 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38887
I20260619 02:41:17.945145 16097 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:17.945355 16097 heartbeater.cc:511] Master 127.7.137.62:38887 requested a full tablet report, sending...
I20260619 02:41:17.945987 15906 ts_manager.cc:194] Registered new tserver with Master: 506031e8d12444dda4c4d5f75bcfccad (127.7.137.1:43953)
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
I20260619 02:41:17.946992 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:38175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:41:17.963721 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:18.044246 16102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.044487 16102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.044523 16102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.045693 16102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.045761 16102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.045804 16102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:18.047201 16102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.047500 16102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.047731 16102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.052754 16108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.052699 16110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.052789 16102 server_base.cc:1034] running on GCE node
W20260619 02:41:18.052743 16107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.053121 16102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.053372 16102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.054533 16102 hybrid_clock.cc:648] HybridClock initialized: now 1781836878054517 us; error 28 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.056962 16102 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:41:18.057453 16102 webserver.cc:460] Webserver started at http://127.7.137.2:41909/ using document root <none> and password file <none>
I20260619 02:41:18.057646 16102 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.057686 16102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.057770 16102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:18.059007 16102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "e8d0c07486ad4b5c8b3c563b6cb6fbf9"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "70c95292b61c4db052216195d5e44c57"
server_key_iv: "b1b96aaa58b699527a608ad5d419939b"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.059319 16102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "e8d0c07486ad4b5c8b3c563b6cb6fbf9"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "70c95292b61c4db052216195d5e44c57"
server_key_iv: "b1b96aaa58b699527a608ad5d419939b"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.060696 16102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:18.061473 16117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.061623 16102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:18.061707 16102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "e8d0c07486ad4b5c8b3c563b6cb6fbf9"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "70c95292b61c4db052216195d5e44c57"
server_key_iv: "b1b96aaa58b699527a608ad5d419939b"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.061759 16102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.090018 16102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.092020 16102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.092139 16102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.092351 16102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.092695 16102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.092747 16102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.092799 16102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.092826 16102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.098321 16102 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:46061
I20260619 02:41:18.098364 16230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:46061 every 8 connection(s)
I20260619 02:41:18.098680 16102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:18.107563 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.099376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:52917 (local address 127.7.137.62:38887)
0619 02:41:18.099442 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.099443 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.099665 (+ 222us) server_negotiation.cc:407] Connection header received
0619 02:41:18.100148 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.100150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.100159 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.100216 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.101267 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.101752 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.102744 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.102920 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.102969 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.105061 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.105062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.105062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.105069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.106276 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.106635 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.106635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.106636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.106687 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.106921 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.106922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.106923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.107047 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.107127 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.107381 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.107459 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:41:18.107741 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.099356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:52917)
0619 02:41:18.099537 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.099579 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.100068 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.100290 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.100294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.100994 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.101188 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.101193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.101871 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.101873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.102669 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.102672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.102989 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.102989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.103086 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.103535 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.103544 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.104969 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:18.106346 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.106352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.106353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.106569 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.106744 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.106744 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.106745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.106836 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.107118 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:18.107124 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:18.107250 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.107357 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.107512 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260619 02:41:18.108188 16231 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38887
I20260619 02:41:18.108266 16231 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:18.108481 16231 heartbeater.cc:511] Master 127.7.137.62:38887 requested a full tablet report, sending...
I20260619 02:41:18.108897 15906 ts_manager.cc:194] Registered new tserver with Master: e8d0c07486ad4b5c8b3c563b6cb6fbf9 (127.7.137.2:46061)
I20260619 02:41:18.108918 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16102
I20260619 02:41:18.109031 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:18.109225 7716 external_mini_cluster.cc:1428] Setting key 5ae378b89c36679a780b4bbfffce667d
I20260619 02:41:18.109417 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:52917
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:41:18.131654 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:18.213038 16236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.213241 16236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.213263 16236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.214450 16236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.214493 16236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.214519 16236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:18.215853 16236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.216027 16236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.216194 16236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.221038 16244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.221063 16242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.221016 16241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.221308 16236 server_base.cc:1034] running on GCE node
I20260619 02:41:18.221534 16236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.221781 16236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.222940 16236 hybrid_clock.cc:648] HybridClock initialized: now 1781836878222928 us; error 25 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.225453 16236 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:41:18.225826 16236 webserver.cc:460] Webserver started at http://127.7.137.3:36671/ using document root <none> and password file <none>
I20260619 02:41:18.226063 16236 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.226107 16236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.226194 16236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:18.227166 16236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "aff7530078a546c0bdea65fd5aefea21"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "3500a0adbcc8efb0df1cd5b56bc28a2f"
server_key_iv: "c90364598610e44a4617fb50ea21cf2e"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.227455 16236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "aff7530078a546c0bdea65fd5aefea21"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "3500a0adbcc8efb0df1cd5b56bc28a2f"
server_key_iv: "c90364598610e44a4617fb50ea21cf2e"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.228801 16236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:18.229557 16251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.229763 16236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:18.229849 16236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "aff7530078a546c0bdea65fd5aefea21"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "3500a0adbcc8efb0df1cd5b56bc28a2f"
server_key_iv: "c90364598610e44a4617fb50ea21cf2e"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.229946 16236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.249723 16236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.251664 16236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.251768 16236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.251932 16236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.252166 16236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.252193 16236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.252215 16236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.252228 16236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.258020 16236 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:41499
I20260619 02:41:18.258097 16364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:41499 every 8 connection(s)
I20260619 02:41:18.258527 16236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:18.266551 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16236
I20260619 02:41:18.266667 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:18.266863 7716 external_mini_cluster.cc:1428] Setting key 1f2a8a8796e2c59af536ff9f41e8a005
I20260619 02:41:18.267699 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.259313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:46635 (local address 127.7.137.62:38887)
0619 02:41:18.259375 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.259378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.259552 (+ 174us) server_negotiation.cc:407] Connection header received
0619 02:41:18.260034 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.260035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.260045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.260088 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.261137 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.261629 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.262590 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.262776 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.262833 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.264983 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.264984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.264985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.264992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.266203 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.266672 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.266673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.266674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.266738 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.267004 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.267005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.267006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.267162 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.267231 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.267532 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.267611 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:41:18.267899 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.259263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:46635)
0619 02:41:18.259427 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.259463 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.259964 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.260183 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.260188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.260863 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.261056 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.261060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.261704 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.261705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.262511 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.262513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.262874 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.262875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.262972 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.263427 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.263436 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.264903 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:18.266286 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.266293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.266295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.266569 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.266816 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:18.266817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:18.266817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:18.266948 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:18.267219 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:18.267225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:18.267392 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 02:41:18.267496 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:41:18.267644 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260619 02:41:18.268342 16365 heartbeater.cc:347] Connected to a master server at 127.7.137.62:38887
I20260619 02:41:18.268435 16365 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:18.268599 16365 heartbeater.cc:511] Master 127.7.137.62:38887 requested a full tablet report, sending...
I20260619 02:41:18.268978 15906 ts_manager.cc:194] Registered new tserver with Master: aff7530078a546c0bdea65fd5aefea21 (127.7.137.3:41499)
I20260619 02:41:18.269299 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:46635
I20260619 02:41:18.278841 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
Entry for principal oryx/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab.
WARNING: no policy specified for oryx/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
Entry for principal oryx/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab.
WARNING: no policy specified for oryx/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
Entry for principal oryx/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab.
I20260619 02:41:18.343461 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15968
I20260619 02:41:18.348538 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16102
I20260619 02:41:18.352684 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16236
I20260619 02:41:18.356841 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 15874
I20260619 02:41:18.361814 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38887
--webserver_interface=127.7.137.62
--webserver_port=39227
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:38887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:18.443816 16374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.444059 16374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.444096 16374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.445258 16374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:18.445327 16374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.445351 16374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.445381 16374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:18.445405 16374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:18.446857 16374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:38887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:38887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=39227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.447130 16374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.447351 16374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.452033 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.452211 16374 server_base.cc:1034] running on GCE node
W20260619 02:41:18.452040 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.452036 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.452481 16374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.452809 16374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.453964 16374 hybrid_clock.cc:648] HybridClock initialized: now 1781836878453950 us; error 35 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.456530 16374 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:18.457280 16374 webserver.cc:460] Webserver started at http://127.7.137.62:39227/ using document root <none> and password file <none>
I20260619 02:41:18.457532 16374 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.457602 16374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.459055 16374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:18.459823 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.460000 16374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:18.460099 16374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "66709428970c3408c2a089cd678d06ec"
server_key_iv: "8b3c97ab65a18816a71016792000241b"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.460484 16374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.467707 16374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.470170 16374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.470345 16374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.474548 16374 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:38887
I20260619 02:41:18.474575 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:38887 every 8 connection(s)
I20260619 02:41:18.474962 16374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:18.476037 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16374
I20260619 02:41:18.476357 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:43953
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:38887
--webserver_port=41549
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:41:18.476480 16442 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:41:18.477188 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Bootstrap starting.
I20260619 02:41:18.485162 16442 log.cc:824] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:18.486806 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:41:18.487095 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Bootstrap complete.
I20260619 02:41:18.489410 16442 raft_consensus.cc:348] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:18.489668 16442 raft_consensus.cc:729] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14496fc04fb4bfb8baf7eaf03d2c3f7, State: Initialized, Role: FOLLOWER
I20260619 02:41:18.489876 16442 consensus_queue.cc:260] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:18.489974 16442 raft_consensus.cc:388] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:18.490029 16442 raft_consensus.cc:482] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:18.490077 16442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:41:18.491077 16442 raft_consensus.cc:504] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:18.491197 16442 leader_election.cc:302] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b14496fc04fb4bfb8baf7eaf03d2c3f7; no voters:
I20260619 02:41:18.491400 16442 leader_election.cc:288] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:41:18.491432 16446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:41:18.491545 16446 raft_consensus.cc:686] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [term 2 LEADER]: Becoming Leader. State: Replica: b14496fc04fb4bfb8baf7eaf03d2c3f7, State: Running, Role: LEADER
I20260619 02:41:18.491634 16446 consensus_queue.cc:237] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } }
I20260619 02:41:18.491678 16442 sys_catalog.cc:564] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:18.491888 16448 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b14496fc04fb4bfb8baf7eaf03d2c3f7. Latest consensus state: current_term: 2 leader_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } } }
I20260619 02:41:18.491982 16448 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:18.492113 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14496fc04fb4bfb8baf7eaf03d2c3f7" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 38887 } } }
I20260619 02:41:18.492165 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:18.492285 16450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:18.492803 16450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:18.493052 16450 catalog_manager.cc:1259] Loaded cluster ID: aa415961ab8745819a3e47b392f43361
I20260619 02:41:18.493221 16450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:18.494398 16450 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:18.494589 16450 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b14496fc04fb4bfb8baf7eaf03d2c3f7: Loaded TSK: 0
I20260619 02:41:18.494812 16450 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:41:18.557878 16444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.558112 16444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.558154 16444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.559301 16444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.559368 16444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.559415 16444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:18.561088 16444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.1.keytab
--principal=oryx/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:43953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=41549
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.561352 16444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.561571 16444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.566757 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.566902 16444 server_base.cc:1034] running on GCE node
W20260619 02:41:18.566815 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.566772 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.567193 16444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.567477 16444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.568638 16444 hybrid_clock.cc:648] HybridClock initialized: now 1781836878568629 us; error 27 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.571398 16444 init.cc:377] Logged in from keytab as oryx/127.7.137.1@KRBTEST.COM (short username oryx)
I20260619 02:41:18.571959 16444 webserver.cc:460] Webserver started at http://127.7.137.1:41549/ using document root <none> and password file <none>
I20260619 02:41:18.572239 16444 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.572324 16444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.575137 16444 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:18.575897 16478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.576159 16444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:18.576256 16444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "506031e8d12444dda4c4d5f75bcfccad"
format_stamp: "Formatted at 2026-06-19 02:41:17 on dist-test-slave-pdnb"
server_key: "07a62aeb0a0e52a3964ef842b746928c"
server_key_iv: "085e58e096b93a0b67607a89a3738ea3"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.576751 16444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.627017 16444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.629427 16444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.629604 16444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.629853 16444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.630184 16444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.630241 16444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.630285 16444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.630312 16444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.636113 16444 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:43953
I20260619 02:41:18.636158 16591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:43953 every 8 connection(s)
I20260619 02:41:18.636503 16444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:18.641891 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16444
I20260619 02:41:18.642190 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:46061
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:38887
--webserver_port=41909
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.645316 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.637244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:39059)
0619 02:41:18.637419 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.637486 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.638011 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.638450 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.638454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.639031 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.639216 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.639221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.640330 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.640331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.641161 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.641164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.641780 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.641781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.641890 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.642484 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.642492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.645084 (+ 2592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
W20260619 02:41:18.645340 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.637327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:39059 (local address 127.7.137.62:38887)
0619 02:41:18.637516 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.637526 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.637572 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:41:18.638224 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.638229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.638269 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.638409 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.639358 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.640224 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.641253 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.641693 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.641809 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.645207 (+ 3398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:39059: BlockingRecv error: recv got EOF from 127.7.137.1:39059 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
W20260619 02:41:18.645712 16592 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:38887 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.137.62:38887: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.650820 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.645979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:58051)
0619 02:41:18.646029 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.646041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.646083 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.646253 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.646253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.646361 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.646404 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.646405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.647366 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.647366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.647853 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.647854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.648361 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.648362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.648448 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.648964 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.648973 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.650771 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:41:18.650974 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.646063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:58051 (local address 127.7.137.62:38887)
0619 02:41:18.646122 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.646124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.646133 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:18.646157 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.646158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.646173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.646236 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.646471 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.647270 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.647929 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.648283 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.648368 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.650890 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:58051: BlockingRecv error: recv got EOF from 127.7.137.1:58051 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.655570 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.651267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:53015)
0619 02:41:18.651366 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.651375 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.651416 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.651538 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.651539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.651640 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.651664 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.651665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.652580 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.652581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.653107 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.653108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.653557 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.653557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.653591 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.653999 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.654003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.655525 (+ 1522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260619 02:41:18.655678 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.651324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53015 (local address 127.7.137.62:38887)
0619 02:41:18.651368 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.651370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.651408 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:41:18.651458 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.651459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.651471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.651543 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.651728 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.652496 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.653178 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.653484 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.653554 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.655617 (+ 2063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:53015: BlockingRecv error: recv got EOF from 127.7.137.1:53015 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:41:18.655738 16592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:41:18.726684 16596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.726977 16596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.727020 16596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.729087 16596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.729146 16596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.729195 16596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:18.731451 16596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.2.keytab
--principal=oryx/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:46061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=41909
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.731652 16596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.731895 16596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.737265 16604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.737278 16601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.737386 16602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.737538 16596 server_base.cc:1034] running on GCE node
I20260619 02:41:18.737746 16596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.737988 16596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.739131 16596 hybrid_clock.cc:648] HybridClock initialized: now 1781836878739110 us; error 37 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.741647 16596 init.cc:377] Logged in from keytab as oryx/127.7.137.2@KRBTEST.COM (short username oryx)
I20260619 02:41:18.742074 16596 webserver.cc:460] Webserver started at http://127.7.137.2:41909/ using document root <none> and password file <none>
I20260619 02:41:18.742317 16596 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.742393 16596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.744364 16596 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:18.745141 16611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.745342 16596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:41:18.745442 16596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "e8d0c07486ad4b5c8b3c563b6cb6fbf9"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "70c95292b61c4db052216195d5e44c57"
server_key_iv: "b1b96aaa58b699527a608ad5d419939b"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.745932 16596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.760058 16596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.762151 16596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.762257 16596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.762423 16596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.762722 16596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.762794 16596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.762827 16596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.762868 16596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.768643 16596 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:46061
I20260619 02:41:18.768761 16724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:46061 every 8 connection(s)
I20260619 02:41:18.769080 16596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:41:18.776152 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.769758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:46583 (local address 127.7.137.62:38887)
0619 02:41:18.769799 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.769801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.770011 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:41:18.770489 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.770491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.770505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.770557 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.771443 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.771946 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.772998 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.773189 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.773250 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.776096 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:46583: BlockingRecv error: recv got EOF from 127.7.137.2:46583 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:18.776212 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.769746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:46583)
0619 02:41:18.769908 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.769940 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.770397 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.770609 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.770612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.771147 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.771335 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.771339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.772031 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.772032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.772900 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.772903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.773265 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.773266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.773363 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.773831 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.773840 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.776015 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
W20260619 02:41:18.776451 16725 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:38887 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.137.62:38887: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
I20260619 02:41:18.777689 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16596
I20260619 02:41:18.777972 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:41499
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:38887
--webserver_port=36671
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.781034 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.776673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:38029)
0619 02:41:18.776759 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.776769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.776818 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.776966 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.776966 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.777098 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.777146 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.777147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.777781 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.777782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.778408 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.778410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.778765 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.778766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.778837 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.779265 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.779270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.780955 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:41:18.781085 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.776757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:38029 (local address 127.7.137.62:38887)
0619 02:41:18.776798 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.776800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.776809 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:18.776886 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.776887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.776896 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.776963 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.777229 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.777703 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.778499 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.778688 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.778766 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.781022 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:38029: BlockingRecv error: recv got EOF from 127.7.137.2:38029 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.785293 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.781281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:55005)
0619 02:41:18.781323 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.781341 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.781377 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.781492 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.781493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.781616 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.781638 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.781639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.782258 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.782259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.782744 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.782745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.783103 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.783104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.783153 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.783545 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.783550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.785206 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:41:18.785311 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.781377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:55005 (local address 127.7.137.62:38887)
0619 02:41:18.781408 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.781410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.781418 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:18.781436 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.781437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.781445 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.781491 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.781707 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.782154 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.782841 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.783023 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.783103 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.785273 (+ 2170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:55005: BlockingRecv error: recv got EOF from 127.7.137.2:55005 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:41:18.785413 16725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:41:18.858297 16728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:18.858493 16728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:18.858515 16728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:18.859688 16728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:18.859728 16728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:18.859786 16728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:18.861438 16728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/krb5kdc/oryx_127.7.137.3.keytab
--principal=oryx/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:41499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=36671
--tserver_master_addrs=127.7.137.62:38887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:18.861666 16728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:18.861899 16728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:18.867105 16733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.867161 16734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:18.867290 16736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:18.867172 16728 server_base.cc:1034] running on GCE node
I20260619 02:41:18.867609 16728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:18.867856 16728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:18.869014 16728 hybrid_clock.cc:648] HybridClock initialized: now 1781836878869011 us; error 37 us; skew 500 ppm
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836878, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:18.871331 16728 init.cc:377] Logged in from keytab as oryx/127.7.137.3@KRBTEST.COM (short username oryx)
I20260619 02:41:18.871800 16728 webserver.cc:460] Webserver started at http://127.7.137.3:36671/ using document root <none> and password file <none>
I20260619 02:41:18.872021 16728 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:18.872099 16728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:18.873492 16728 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:18.874033 16743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.874181 16728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:18.874280 16728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "aff7530078a546c0bdea65fd5aefea21"
format_stamp: "Formatted at 2026-06-19 02:41:18 on dist-test-slave-pdnb"
server_key: "3500a0adbcc8efb0df1cd5b56bc28a2f"
server_key_iv: "c90364598610e44a4617fb50ea21cf2e"
server_key_version: "encryptionkey@0"
I20260619 02:41:18.874619 16728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:18.900700 16728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:18.902724 16728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:18.902896 16728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:18.903132 16728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:18.903450 16728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:18.903512 16728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.903553 16728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:18.903579 16728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:18.909198 16728 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:41499
I20260619 02:41:18.909245 16856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:41499 every 8 connection(s)
I20260619 02:41:18.909597 16728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:18.913148 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16728
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:41:18.917165 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.910304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:40229 (local address 127.7.137.62:38887)
0619 02:41:18.910372 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.910375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.910611 (+ 236us) server_negotiation.cc:407] Connection header received
0619 02:41:18.911086 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.911087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.911098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.911149 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.912054 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.912532 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.913797 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.914065 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.914114 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.917112 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:40229: BlockingRecv error: recv got EOF from 127.7.137.3:40229 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:41:18.917263 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.910311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:40229)
0619 02:41:18.910492 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.910527 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.911002 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.911217 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.911220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.911770 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.911960 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.911980 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.912608 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.912608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.913668 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.913671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.914146 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.914147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.914305 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.914859 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.914867 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.917023 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
W20260619 02:41:18.917462 16857 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:38887 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.137.62:38887: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
I20260619 02:41:18.921059 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.913636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.7.137.62:38887)
0619 02:41:18.913747 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.913748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.913756 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:18.913839 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.913839 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.913850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.913916 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.914642 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.915116 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.915902 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.916115 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.916199 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.917486 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:18.917491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:18.917491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:18.917503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:18.919434 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.919830 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.919833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.919833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.919924 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:18.920153 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:18.920154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:18.920154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:18.920461 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:18.920602 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:18.920827 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:41:18.920942 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.922640 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.917665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:60799)
0619 02:41:18.917706 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.917722 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.917777 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.918260 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.918261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.918440 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.918484 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.918486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.919150 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.919150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.919808 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.919811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.920158 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.920159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.920230 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.920631 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.920636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.922597 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:41:18.922744 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.917715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:60799 (local address 127.7.137.62:38887)
0619 02:41:18.917998 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.918000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.918006 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:41:18.918022 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.918022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.918030 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.918255 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.918594 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.919059 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.919889 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.920073 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.920156 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.922676 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:60799: BlockingRecv error: recv got EOF from 127.7.137.3:60799 (error 108)
Metrics: {"server-negotiator.queue_time_us":266}
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:18 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:18.926751 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:18.922967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:43667)
0619 02:41:18.923024 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:18.923035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:18.923076 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:18.923184 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:18.923185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:18.923288 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:18.923312 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.923313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.923838 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.923838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.924338 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:18.924340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.924667 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.924668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:18.924705 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.925149 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:18.925154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:18.926709 (+ 1555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:41:18.926875 16595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:18.923027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:43667 (local address 127.7.137.62:38887)
0619 02:41:18.923070 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:18.923071 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:18.923078 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:18.923134 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:18.923135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:18.923141 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:18.923183 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:18.923350 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.923768 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.924423 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:18.924601 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:18.924654 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:18.926804 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:43667: BlockingRecv error: recv got EOF from 127.7.137.3:43667 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:41:18.926877 16857 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:19.660928 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:19.656231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:45083)
0619 02:41:19.656395 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:19.656406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:19.656460 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:19.656594 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:19.656595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:19.656826 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:19.656872 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.656874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.657653 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.657654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.658195 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.658197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.658532 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.658532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.658571 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.658995 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:19.658999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:19.660859 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
W20260619 02:41:19.661047 16863 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:19.656320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:45083 (local address 127.7.137.62:38887)
0619 02:41:19.656451 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:19.656453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:19.656462 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:19.656505 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:19.656506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:19.656520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:19.656595 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:19.656980 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.657587 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.658269 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.658467 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.658522 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.660961 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:45083: BlockingRecv error: recv got EOF from 127.7.137.1:45083 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":85,"threads_started":1}
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:19.790426 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:19.785838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:36027)
0619 02:41:19.785986 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:19.785997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:19.786042 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:19.786190 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:19.786192 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:19.786427 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:19.786499 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.786501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.787222 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.787223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.787756 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.787757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.788057 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.788058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.788094 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.788581 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:19.788586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:19.790357 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260619 02:41:19.790496 16863 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:19.785933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:36027 (local address 127.7.137.62:38887)
0619 02:41:19.786017 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:19.786019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:19.786032 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:41:19.786100 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:19.786101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:19.786114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:19.786156 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:19.786579 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.787157 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.787826 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.788000 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.788048 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.790440 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:36027: BlockingRecv error: recv got EOF from 127.7.137.2:36027 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:19 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:19.931947 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:19.927239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:53401)
0619 02:41:19.927374 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:19.927385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:19.927441 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:19.927549 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:19.927550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:19.927706 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:19.927762 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.927763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.928411 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.928412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.929032 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:19.929035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.929395 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.929395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:19.929432 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.929937 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:19.929943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:19.931837 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260619 02:41:19.931985 16863 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:19.927369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:53401 (local address 127.7.137.62:38887)
0619 02:41:19.927430 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:19.927432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:19.927442 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:19.927476 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:19.927477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:19.927489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:19.927546 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:19.927829 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.928341 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.929129 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:19.929329 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:19.929388 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:19.931918 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:53401: BlockingRecv error: recv got EOF from 127.7.137.3:53401 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:20.666427 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:20.661405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:46259)
0619 02:41:20.661567 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:20.661577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:20.661632 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:20.661830 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:20.661831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:20.662002 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:20.662050 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.662052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.662774 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.662775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.663298 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.663299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.663864 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.663865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.663934 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.664474 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:20.664479 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:20.666328 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
W20260619 02:41:20.666474 16867 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:20.661493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:46259 (local address 127.7.137.62:38887)
0619 02:41:20.661644 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:20.661646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:20.661676 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:41:20.661704 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:20.661705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:20.661725 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:20.661795 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:20.662131 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.662641 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.663493 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.663774 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.663861 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.666405 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:46259: BlockingRecv error: recv got EOF from 127.7.137.1:46259 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:20.795730 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:20.790961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:54687)
0619 02:41:20.791114 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:20.791124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:20.791181 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:20.791370 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:20.791371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:20.791636 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:20.791695 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.791699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.792404 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.792404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.793027 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.793030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.793347 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.793349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.793401 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.793870 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:20.793875 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:20.795629 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
W20260619 02:41:20.795763 16867 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:20.791005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:54687 (local address 127.7.137.62:38887)
0619 02:41:20.791119 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:20.791121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:20.791194 (+ 73us) server_negotiation.cc:407] Connection header received
0619 02:41:20.791228 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:20.791229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:20.791243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:20.791313 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:20.791773 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.792299 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.793092 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.793268 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.793330 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.795701 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:54687: BlockingRecv error: recv got EOF from 127.7.137.2:54687 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:20 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:20.937214 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:20.932507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:33207)
0619 02:41:20.932645 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:20.932658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:20.932739 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:20.932957 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:20.932959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:20.933144 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:20.933191 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.933193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.933972 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.933973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.934497 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:20.934499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.934835 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.934836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:20.934878 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.935336 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:20.935340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:20.937152 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
W20260619 02:41:20.937345 16867 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:20.932703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:33207 (local address 127.7.137.62:38887)
0619 02:41:20.932794 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:20.932798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:20.932829 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:41:20.932861 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:20.932861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:20.932876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:20.932937 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:20.933288 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.933901 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.934569 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:20.934743 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:20.934790 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:20.937262 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:33207: BlockingRecv error: recv got EOF from 127.7.137.3:33207 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:21.674790 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:21.666963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:48563)
0619 02:41:21.667117 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:21.667128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:21.667181 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:21.667326 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:21.667327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:21.667490 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:21.667539 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.667541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.668185 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.668185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.668791 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.668794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.669116 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.669117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.669155 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.672918 (+ 3763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:21.672924 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:21.674723 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
W20260619 02:41:21.674919 16871 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:21.667026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:48563 (local address 127.7.137.62:38887)
0619 02:41:21.667157 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:21.667159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:21.667168 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:21.667239 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:21.667240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:21.667250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:21.667309 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:21.667604 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.668108 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.668860 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.669053 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.669106 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.674833 (+ 5727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:48563: BlockingRecv error: recv got EOF from 127.7.137.1:48563 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:21.800628 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:21.796216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:57603)
0619 02:41:21.796352 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:21.796367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:21.796426 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:21.796566 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:21.796566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:21.796776 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:21.796830 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.796832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.797455 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.797455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.797969 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.797971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.798295 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.798296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.798332 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.798780 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:21.798784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:21.800531 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
W20260619 02:41:21.800670 16871 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:21.796275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:57603 (local address 127.7.137.62:38887)
0619 02:41:21.796344 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:21.796346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:21.796424 (+ 78us) server_negotiation.cc:407] Connection header received
0619 02:41:21.796493 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:21.796494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:21.796504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:21.796551 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:21.796893 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.797383 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.798043 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.798230 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.798296 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.800610 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:57603: BlockingRecv error: recv got EOF from 127.7.137.2:57603 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:21 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:21.942025 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:21.937708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:32949)
0619 02:41:21.937823 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:21.937841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:21.937895 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:21.938032 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:21.938032 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:21.938181 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:21.938229 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.938231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.938848 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.938848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.939379 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:21.939380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.939681 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.939682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:21.939719 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.940168 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:21.940172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:21.941959 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
W20260619 02:41:21.942099 16871 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:21.937761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:32949 (local address 127.7.137.62:38887)
0619 02:41:21.937831 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:21.937833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:21.937895 (+ 62us) server_negotiation.cc:407] Connection header received
0619 02:41:21.937960 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:21.937960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:21.937973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:21.938020 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:21.938293 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.938773 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.939447 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:21.939620 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:21.939668 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:21.942052 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:32949: BlockingRecv error: recv got EOF from 127.7.137.3:32949 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:22.680112 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:22.675317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:52201)
0619 02:41:22.675478 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:22.675488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:22.675543 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:22.675683 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:22.675684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:22.675838 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:22.675884 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.675885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.676544 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.676545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.677190 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.677193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.677558 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.677559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.677598 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.678114 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:22.678119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:22.680033 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
W20260619 02:41:22.680251 16875 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:22.675411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:52201 (local address 127.7.137.62:38887)
0619 02:41:22.675536 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:22.675538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:22.675556 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:41:22.675583 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:22.675584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:22.675595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:22.675654 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:22.675960 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.676472 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.677280 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.677486 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.677552 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.680181 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:52201: BlockingRecv error: recv got EOF from 127.7.137.1:52201 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:22.808888 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:22.801102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:42529)
0619 02:41:22.801259 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:22.801283 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:22.801338 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:22.801466 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:22.801467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:22.801626 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:22.801672 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.801673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.802293 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.802294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.802821 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.802823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.803138 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.803139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.803178 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.807006 (+ 3828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:22.807011 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:22.808827 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
W20260619 02:41:22.809000 16875 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:22.801171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:42529 (local address 127.7.137.62:38887)
0619 02:41:22.801225 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:22.801227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:22.801323 (+ 96us) server_negotiation.cc:407] Connection header received
0619 02:41:22.801383 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:22.801383 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:22.801395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:22.801450 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:22.801728 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.802215 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.802889 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.803070 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.803119 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.808927 (+ 5808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:42529: BlockingRecv error: recv got EOF from 127.7.137.2:42529 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:22 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:22.947055 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:22.942522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:38739)
0619 02:41:22.942676 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:22.942686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:22.942745 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:22.942934 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:22.942935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:22.943088 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:22.943135 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.943136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.943795 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.943796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.944314 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:22.944316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.944664 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.944664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:22.944700 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.945252 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:22.945257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:22.946960 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
W20260619 02:41:22.947098 16875 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:22.942580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:38739 (local address 127.7.137.62:38887)
0619 02:41:22.942651 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:22.942654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:22.942755 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:41:22.942846 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:22.942847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:22.942860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:22.942916 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:22.943202 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.943700 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.944399 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:22.944600 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:22.944657 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:22.947036 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:38739: BlockingRecv error: recv got EOF from 127.7.137.3:38739 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:23.685475 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:23.680603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:47179)
0619 02:41:23.680765 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:23.680782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:23.680829 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:23.681058 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:23.681058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:23.681214 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:23.681263 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.681265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.682110 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.682110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.682625 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.682627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.682959 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.682960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.682996 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.683427 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:23.683432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:23.685417 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
W20260619 02:41:23.685612 16879 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:23.680775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:47179 (local address 127.7.137.62:38887)
0619 02:41:23.680917 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:23.680919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:23.680929 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:23.680955 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:23.680956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:23.680967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:23.681051 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:23.681350 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.682036 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.682697 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.682892 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.682949 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.685499 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:47179: BlockingRecv error: recv got EOF from 127.7.137.1:47179 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:23.814021 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:23.809378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:45905)
0619 02:41:23.809522 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:23.809533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:23.809585 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:23.809693 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:23.809693 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:23.809879 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:23.809926 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.809928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.810542 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.810543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.811081 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.811083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.811475 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.811477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.811534 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.812053 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:23.812058 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:23.813945 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
W20260619 02:41:23.814126 16879 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:23.809494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:45905 (local address 127.7.137.62:38887)
0619 02:41:23.809540 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:23.809541 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:23.809570 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:41:23.809628 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:23.809629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:23.809639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:23.809688 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:23.809986 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.810477 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.811147 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.811378 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.811443 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.814034 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:45905: BlockingRecv error: recv got EOF from 127.7.137.2:45905 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:23 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:23.952430 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:23.947588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:35283)
0619 02:41:23.947881 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:23.947893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:23.947991 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:23.948180 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:23.948181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:23.948367 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:23.948412 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.948414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.949155 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.949156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.949758 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:23.949760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.950108 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.950109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:23.950166 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.950592 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:23.950596 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:23.952334 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":83,"threads_started":1}
W20260619 02:41:23.952481 16879 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:23.947682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:35283 (local address 127.7.137.62:38887)
0619 02:41:23.947872 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:23.947874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:23.947964 (+ 90us) server_negotiation.cc:407] Connection header received
0619 02:41:23.948066 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:23.948067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:23.948101 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:23.948189 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:23.948494 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.949075 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.949843 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:23.950037 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:23.950112 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:23.952414 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:35283: BlockingRecv error: recv got EOF from 127.7.137.3:35283 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:24.690554 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:24.685884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:57585)
0619 02:41:24.686032 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:24.686043 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:24.686089 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:24.686227 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:24.686228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:24.686389 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:24.686442 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.686443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.687112 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.687112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.687620 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.687621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.688015 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.688016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.688063 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.688609 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:24.688613 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:24.690456 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260619 02:41:24.690614 16883 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:24.685981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:57585 (local address 127.7.137.62:38887)
0619 02:41:24.686104 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:24.686107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:24.686115 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:24.686135 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:24.686136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:24.686147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:24.686209 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:24.686510 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.687039 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.687726 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.687932 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.687999 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.690541 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:57585: BlockingRecv error: recv got EOF from 127.7.137.1:57585 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:24.818849 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:24.814515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:48755)
0619 02:41:24.814658 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:24.814676 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:24.814734 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:24.814868 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:24.814869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:24.815031 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:24.815079 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.815081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.815694 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.815694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.816219 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.816221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.816584 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.816585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.816623 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.817103 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:24.817108 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:24.818787 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260619 02:41:24.819001 16883 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:24.814585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:48755 (local address 127.7.137.62:38887)
0619 02:41:24.814673 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:24.814675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:24.814762 (+ 87us) server_negotiation.cc:407] Connection header received
0619 02:41:24.814788 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:24.814789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:24.814800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:24.814850 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:24.815141 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.815629 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.816318 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.816511 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.816581 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.818922 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:48755: BlockingRecv error: recv got EOF from 127.7.137.2:48755 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:24 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:24.957602 16885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:24.952976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:51211)
0619 02:41:24.953136 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:24.953148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:24.953217 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:24.953352 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:24.953353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:24.953546 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:24.953592 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.953593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.954258 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.954259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.954795 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:24.954796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.955164 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.955165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:24.955224 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.955683 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:24.955687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:24.957544 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
W20260619 02:41:24.957741 16883 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:24.953044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:51211 (local address 127.7.137.62:38887)
0619 02:41:24.953121 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:24.953123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:24.953199 (+ 76us) server_negotiation.cc:407] Connection header received
0619 02:41:24.953270 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:24.953271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:24.953286 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:24.953342 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:24.953674 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.954175 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.954897 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:24.955090 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:24.955166 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:24.957657 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:51211: BlockingRecv error: recv got EOF from 127.7.137.3:51211 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:25.695837 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:25.691059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:60105)
0619 02:41:25.691215 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:25.691226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:25.691277 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:25.691425 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:25.691426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:25.691668 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:25.691732 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.691735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.692501 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.692502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.693099 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.693101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.693548 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.693549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.693601 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.694061 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:25.694065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:25.695774 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
W20260619 02:41:25.695984 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:25.691153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:60105 (local address 127.7.137.62:38887)
0619 02:41:25.691289 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:25.691290 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:25.691298 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:41:25.691319 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:25.691319 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:25.691330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:25.691394 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:25.691839 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.692426 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.693193 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.693459 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.693525 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.695895 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:60105: BlockingRecv error: recv got EOF from 127.7.137.1:60105 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:25.823823 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:25.819342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:40699)
0619 02:41:25.819463 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:25.819479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:25.819532 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:25.819651 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:25.819652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:25.819814 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:25.819861 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.819863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.820497 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.820498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.821154 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.821157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.821501 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.821502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.821549 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.822028 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:25.822032 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:25.823764 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
W20260619 02:41:25.823925 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:25.819398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:40699 (local address 127.7.137.62:38887)
0619 02:41:25.819482 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:25.819484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:25.819522 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:41:25.819585 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:25.819586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:25.819596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:25.819648 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:25.819924 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.820415 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.821246 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.821434 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.821491 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.823856 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:40699: BlockingRecv error: recv got EOF from 127.7.137.2:40699 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:25 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:25.962843 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:25.958197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:46447)
0619 02:41:25.958345 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:25.958356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:25.958414 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:25.958539 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:25.958540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:25.958881 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:25.958940 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.958943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.959641 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.959642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.960165 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:25.960166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.960481 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.960482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:25.960521 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.961006 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:25.961011 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:25.962785 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260619 02:41:25.962970 16887 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:25.958210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:46447 (local address 127.7.137.62:38887)
0619 02:41:25.958257 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:25.958259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:25.958421 (+ 162us) server_negotiation.cc:407] Connection header received
0619 02:41:25.958459 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:25.958460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:25.958473 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:25.958519 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:25.959057 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.959565 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.960238 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:25.960418 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:25.960469 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:25.962892 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:46447: BlockingRecv error: recv got EOF from 127.7.137.3:46447 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:26.700915 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:26.696354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:35263)
0619 02:41:26.696469 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:26.696481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:26.696542 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:26.696757 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:26.696758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:26.696928 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:26.696975 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.696977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.697654 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.697656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.698187 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.698189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.698518 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.698519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.698564 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.699058 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:26.699063 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:26.700857 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
W20260619 02:41:26.701069 16891 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:26.696444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:35263 (local address 127.7.137.62:38887)
0619 02:41:26.696590 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:26.696593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:26.696600 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:26.696629 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:26.696630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:26.696639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:26.696695 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:26.697045 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.697532 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.698265 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.698448 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.698496 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.700984 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:35263: BlockingRecv error: recv got EOF from 127.7.137.1:35263 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:26.829301 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:26.824358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:56859)
0619 02:41:26.824497 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:26.824508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:26.824557 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:26.824753 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:26.824754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:26.824917 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:26.824975 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.824977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.825778 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.825779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.826557 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.826560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.826901 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.826902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.826942 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.827396 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:26.827400 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:26.829242 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260619 02:41:26.829433 16891 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:26.824456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:56859 (local address 127.7.137.62:38887)
0619 02:41:26.824537 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:26.824538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:26.824548 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:26.824623 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:26.824623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:26.824635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:26.824682 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:26.825046 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.825675 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.826620 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.826835 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.826889 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.829357 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:56859: BlockingRecv error: recv got EOF from 127.7.137.2:56859 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:26 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:26.968307 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:26.963546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:46789)
0619 02:41:26.963678 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:26.963691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:26.963759 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:26.963910 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:26.963911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:26.964074 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:26.964124 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.964126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.964854 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.964854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.965371 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:26.965372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.965667 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.965668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:26.965706 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.966134 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:26.966139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:26.968243 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260619 02:41:26.968384 16891 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:26.963619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:46789 (local address 127.7.137.62:38887)
0619 02:41:26.963697 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:26.963698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:26.963758 (+ 60us) server_negotiation.cc:407] Connection header received
0619 02:41:26.963802 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:26.963803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:26.963831 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:26.963882 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:26.964183 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.964786 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.965434 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:26.965609 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:26.965656 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:26.968333 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:46789: BlockingRecv error: recv got EOF from 127.7.137.3:46789 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:27.706151 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:27.701460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:44573)
0619 02:41:27.701654 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:27.701670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:27.701730 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:27.701866 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:27.701867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:27.702043 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:27.702092 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.702094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.702750 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.702750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.703300 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.703301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.703726 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.703727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.703781 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.704225 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:27.704229 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:27.706028 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
W20260619 02:41:27.706218 16895 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:27.701539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:44573 (local address 127.7.137.62:38887)
0619 02:41:27.701718 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:27.701720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:27.701730 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:27.701777 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:27.701778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:27.701788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:27.701845 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:27.702159 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.702678 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.703383 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.703618 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.703710 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.706136 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:44573: BlockingRecv error: recv got EOF from 127.7.137.1:44573 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:27.837875 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:27.829788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:56933)
0619 02:41:27.829927 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:27.829939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:27.830002 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:27.830164 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:27.830165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:27.830365 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:27.830411 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.830413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.831093 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.831094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.831631 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.831632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.832022 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.832023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.832077 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.835666 (+ 3589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:27.835671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:27.837814 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260619 02:41:27.838025 16895 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:27.829925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:56933 (local address 127.7.137.62:38887)
0619 02:41:27.829980 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:27.829981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:27.829992 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:41:27.830062 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:27.830063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:27.830076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:27.830152 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:27.830494 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.831008 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.831758 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.831944 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.832024 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.837946 (+ 5922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:56933: BlockingRecv error: recv got EOF from 127.7.137.2:56933 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:27 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:27.973593 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:27.968967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:56539)
0619 02:41:27.969109 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:27.969128 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:27.969209 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:27.969399 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:27.969400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:27.969621 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:27.969676 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.969677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.970323 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.970324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.970845 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:27.970846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.971166 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.971166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:27.971206 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.971672 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:27.971677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:27.973535 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
W20260619 02:41:27.973757 16895 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:27.969036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:56539 (local address 127.7.137.62:38887)
0619 02:41:27.969100 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:27.969102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:27.969196 (+ 94us) server_negotiation.cc:407] Connection header received
0619 02:41:27.969286 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:27.969287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:27.969304 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:27.969369 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:27.969752 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.970256 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.970916 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:27.971097 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:27.971151 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:27.973686 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:56539: BlockingRecv error: recv got EOF from 127.7.137.3:56539 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:28.711817 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:28.706859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:50931)
0619 02:41:28.707025 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:28.707038 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:28.707101 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:28.707304 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:28.707305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:28.707477 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:28.707527 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.707528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.708353 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.708354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.708935 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.708937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.709364 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.709365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.709431 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.709914 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:28.709919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:28.711756 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":61,"threads_started":1}
W20260619 02:41:28.712085 16899 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:28.707031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:50931 (local address 127.7.137.62:38887)
0619 02:41:28.707153 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:28.707155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:28.707166 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:41:28.707200 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:28.707200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:28.707211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:28.707271 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:28.707644 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.708257 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.709052 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.709271 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.709361 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.711987 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:50931: BlockingRecv error: recv got EOF from 127.7.137.1:50931 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:28.843071 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:28.838301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:60375)
0619 02:41:28.838489 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:28.838501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:28.838574 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:28.838710 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:28.838711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:28.838907 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:28.838951 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.838953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.839632 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.839633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.840167 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.840168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.840563 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.840564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.840633 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.841137 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:28.841141 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:28.842966 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
W20260619 02:41:28.843108 16899 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:28.838411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:60375 (local address 127.7.137.62:38887)
0619 02:41:28.838485 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:28.838488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:28.838555 (+ 67us) server_negotiation.cc:407] Connection header received
0619 02:41:28.838634 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:28.838635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:28.838649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:28.838696 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:28.839038 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.839544 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.840277 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.840476 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.840562 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.843045 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:60375: BlockingRecv error: recv got EOF from 127.7.137.2:60375 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:28 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:28.979177 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:28.974047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:36357)
0619 02:41:28.974233 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:28.974244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:28.974328 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:28.974504 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:28.974504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:28.974724 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:28.974772 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.974773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.975491 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.975492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.976040 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:28.976041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.976484 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.976485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:28.976551 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.977023 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:28.977028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:28.979118 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
W20260619 02:41:28.979358 16899 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:28.974251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:36357 (local address 127.7.137.62:38887)
0619 02:41:28.974317 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:28.974319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:28.974329 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:28.974398 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:28.974399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:28.974414 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:28.974497 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:28.974857 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.975389 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.976142 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:28.976355 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:28.976482 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:28.979292 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:36357: BlockingRecv error: recv got EOF from 127.7.137.3:36357 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:29.717483 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:29.712420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:37209)
0619 02:41:29.712633 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:29.712646 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:29.712712 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:29.712908 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:29.712909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:29.713074 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:29.713120 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.713121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.713800 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.713802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.714362 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.714365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.714791 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.714792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.714862 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.715502 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:29.715508 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:29.717408 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":115,"threads_started":1}
W20260619 02:41:29.717607 16903 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:29.712535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:37209 (local address 127.7.137.62:38887)
0619 02:41:29.712692 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:29.712694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:29.712704 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:29.712810 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:29.712811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:29.712823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:29.712884 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:29.713188 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.713703 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.714443 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.714693 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.714749 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.717500 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:37209: BlockingRecv error: recv got EOF from 127.7.137.1:37209 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":56,"threads_started":1}
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:29.848415 16904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:29.843761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:33031)
0619 02:41:29.843945 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:29.843956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:29.844048 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:29.844210 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:29.844212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:29.844457 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:29.844516 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.844518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.845167 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.845168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.845724 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.845726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.846046 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.846047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.846087 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.846520 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:29.846524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:29.848302 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
W20260619 02:41:29.848456 16903 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:29.843854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:33031 (local address 127.7.137.62:38887)
0619 02:41:29.843925 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:29.843928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:29.844035 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:41:29.844109 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:29.844110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:29.844123 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:29.844178 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:29.844591 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.845101 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.845795 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.845985 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.846034 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.848390 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:33031: BlockingRecv error: recv got EOF from 127.7.137.2:33031 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:29 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:29.984180 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:29.979595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:54791)
0619 02:41:29.979756 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:29.979767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:29.979831 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:29.979950 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:29.979951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:29.980109 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:29.980154 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.980155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.980825 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.980825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.981382 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:29.981384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.981717 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.981718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:29.981767 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.982242 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:29.982246 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:29.984112 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
W20260619 02:41:29.984313 16903 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:29.979648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:54791 (local address 127.7.137.62:38887)
0619 02:41:29.979713 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:29.979715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:29.979830 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:41:29.979874 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:29.979875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:29.979888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:29.979937 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:29.980215 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.980759 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.981460 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:29.981636 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:29.981691 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:29.984232 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:54791: BlockingRecv error: recv got EOF from 127.7.137.3:54791 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:30.722802 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:30.718002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:53637)
0619 02:41:30.718188 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:30.718199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:30.718256 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:30.718390 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:30.718390 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:30.718593 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:30.718652 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.718654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.719501 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.719501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.720029 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.720030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.720353 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.720354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.720392 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.720908 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:30.720912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:30.722747 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":110,"threads_started":1}
W20260619 02:41:30.722930 16907 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:30.718131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53637 (local address 127.7.137.62:38887)
0619 02:41:30.718258 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:30.718260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:30.718287 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:41:30.718310 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:30.718311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:30.718321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:30.718371 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:30.718752 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.719434 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.720097 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.720280 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.720339 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.722841 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:53637: BlockingRecv error: recv got EOF from 127.7.137.1:53637 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":51,"threads_started":1}
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:30.853591 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:30.848920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:50159)
0619 02:41:30.849046 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:30.849059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:30.849131 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:30.849258 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:30.849258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:30.849437 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:30.849482 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.849484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.850181 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.850182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.850773 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.850776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.851145 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.851146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.851201 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.851626 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:30.851630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:30.853474 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
W20260619 02:41:30.853610 16907 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:30.849027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:50159 (local address 127.7.137.62:38887)
0619 02:41:30.849074 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:30.849076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:30.849114 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:41:30.849178 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:30.849178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:30.849189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:30.849234 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:30.849544 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.850107 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.850864 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.851067 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.851148 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.853552 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:50159: BlockingRecv error: recv got EOF from 127.7.137.2:50159 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:30 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:30.989241 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:30.984653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:52699)
0619 02:41:30.984812 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:30.984825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:30.984894 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:30.985050 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:30.985051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:30.985208 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:30.985254 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.985256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.985898 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.985898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.986417 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:30.986419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.986847 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.986848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:30.986890 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.987386 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:30.987396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:30.989182 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
W20260619 02:41:30.989399 16907 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:30.984792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:52699 (local address 127.7.137.62:38887)
0619 02:41:30.984859 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:30.984860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:30.984871 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:41:30.984950 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:30.984951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:30.984963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:30.985013 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:30.985329 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.985831 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.986495 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:30.986783 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:30.986840 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:30.989279 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:52699: BlockingRecv error: recv got EOF from 127.7.137.3:52699 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:31.728000 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:31.723298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:45597)
0619 02:41:31.723480 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:31.723492 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:31.723554 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:31.723703 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:31.723704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:31.723873 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:31.723917 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.723919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.724545 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.724546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.725191 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.725194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.725522 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.725523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.725561 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.726081 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:31.726085 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:31.727890 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":117,"threads_started":1}
W20260619 02:41:31.728035 16911 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:31.723391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:45597 (local address 127.7.137.62:38887)
0619 02:41:31.723493 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:31.723495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:31.723556 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:41:31.723609 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:31.723610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:31.723621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:31.723670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:31.723984 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.724475 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.725271 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.725460 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.725512 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.727964 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:45597: BlockingRecv error: recv got EOF from 127.7.137.1:45597 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:31.858624 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:31.854085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:59191)
0619 02:41:31.854237 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:31.854251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:31.854316 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:31.854495 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:31.854496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:31.854669 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:31.854716 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.854718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.855384 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.855385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.855979 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.855981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.856305 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.856306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.856347 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.856870 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:31.856875 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:31.858553 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
W20260619 02:41:31.858695 16911 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:31.854269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:59191 (local address 127.7.137.62:38887)
0619 02:41:31.854343 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:31.854349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:31.854358 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:31.854384 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:31.854385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:31.854395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:31.854446 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:31.854793 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.855293 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.856058 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.856239 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.856300 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.858649 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:59191: BlockingRecv error: recv got EOF from 127.7.137.2:59191 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:31 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:31.994565 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:31.989728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:39913)
0619 02:41:31.989897 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:31.989908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:31.989968 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:31.990126 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:31.990127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:31.990321 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:31.990364 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.990366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.991040 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.991041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.991691 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:31.991694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.992075 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.992076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:31.992132 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.992607 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:31.992612 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:31.994495 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
W20260619 02:41:31.994673 16911 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:31.989817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:39913 (local address 127.7.137.62:38887)
0619 02:41:31.989886 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:31.989888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:31.989949 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:41:31.990058 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:31.990059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:31.990072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:31.990123 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:31.990444 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.990945 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.991780 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:31.991980 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:31.992077 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:31.994586 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:39913: BlockingRecv error: recv got EOF from 127.7.137.3:39913 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:32.732980 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:32.728474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:51753)
0619 02:41:32.728619 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:32.728630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:32.728695 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:32.728875 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:32.728876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:32.729057 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:32.729105 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.729107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.729724 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.729724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.730275 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.730277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.730583 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.730583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.730647 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.731105 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:32.731109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:32.732870 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
W20260619 02:41:32.733021 16915 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:32.728527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:51753 (local address 127.7.137.62:38887)
0619 02:41:32.728647 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:32.728649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:32.728698 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:41:32.728777 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:32.728778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:32.728790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:32.728859 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:32.729168 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.729657 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.730342 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.730523 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.730572 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.732948 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:51753: BlockingRecv error: recv got EOF from 127.7.137.1:51753 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:32.867772 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:32.859147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:42161)
0619 02:41:32.859341 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:32.859354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:32.859441 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:32.859592 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:32.859593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:32.859815 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:32.859859 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.859861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.860566 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.860567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.861183 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.861185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.861551 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.861552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.861606 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.865934 (+ 4328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:32.865939 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:32.867675 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":107,"threads_started":1}
W20260619 02:41:32.867823 16915 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:32.859243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:42161 (local address 127.7.137.62:38887)
0619 02:41:32.859322 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:32.859324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:32.859426 (+ 102us) server_negotiation.cc:407] Connection header received
0619 02:41:32.859499 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:32.859500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:32.859512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:32.859585 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:32.859951 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.860463 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.861268 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.861465 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.861550 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.867760 (+ 6210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:42161: BlockingRecv error: recv got EOF from 127.7.137.2:42161 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:32 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.3@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:32.999552 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:32.995004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.3:54829)
0619 02:41:32.995113 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:32.995131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:32.995190 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:32.995326 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:32.995326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:32.995487 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:32.995533 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.995534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.996148 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.996149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.996668 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:32.996669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.997024 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.997025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:32.997066 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.997631 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:32.997636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:32.999482 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
W20260619 02:41:32.999673 16915 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:32.995074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:54829 (local address 127.7.137.62:38887)
0619 02:41:32.995151 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:32.995153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:32.995174 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:41:32.995245 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:32.995246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:32.995259 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:32.995306 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:32.995591 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.996070 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.996782 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:32.996962 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:32.997013 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:32.999586 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.3:54829: BlockingRecv error: recv got EOF from 127.7.137.3:54829 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 02:41:33 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:33 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.1@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:33.737758 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:33.733500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.1:56065)
0619 02:41:33.733645 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:33.733661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:33.733704 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:33.733816 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:33.733816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:33.733970 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:33.734025 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:33.734026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.734675 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.734676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:33.735201 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:33.735203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.735539 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.735540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:33.735578 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:33.736002 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:33.736006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:33.737686 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260619 02:41:33.737825 16919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:33.733579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:56065 (local address 127.7.137.62:38887)
0619 02:41:33.733714 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:33.733715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:33.733722 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:41:33.733743 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:33.733744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:33.733754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:33.733811 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:33.734088 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.734605 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.735280 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.735474 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.735529 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:33.737774 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.1:56065: BlockingRecv error: recv got EOF from 127.7.137.1:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
Jun 19 02:41:33 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:41:33 dist-test-slave-pdnb krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.137.2@KRBTEST.COM for oryx/127.7.137.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:41:33.873052 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:33.868356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:38887 (local address 127.7.137.2:55083)
0619 02:41:33.868624 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:33.868634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:33.868691 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:33.868870 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:33.868870 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:33.869025 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:33.869071 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:33.869073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.869695 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.869695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:33.870232 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:33.870234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.870545 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.870546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:33.870584 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:33.871101 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:33.871107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:33.872982 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.137.62:38887: Server oryx/127.7.137.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":178,"threads_started":1}
W20260619 02:41:33.873155 16919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:33.868441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:55083 (local address 127.7.137.62:38887)
0619 02:41:33.868511 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:33.868513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:33.868693 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:41:33.868776 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:33.868777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:33.868794 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:33.868851 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:33.869133 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.869624 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.870305 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:33.870482 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:33.870533 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:33.873066 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.137.2:55083: BlockingRecv error: recv got EOF from 127.7.137.2:55083 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:41:33.914341 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16444
I20260619 02:41:33.918923 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16596
I20260619 02:41:33.923154 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16728
I20260619 02:41:33.927114 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16374
2026-06-19T02:41:33Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17442 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T02:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:33Z Disabled control of system clock
I20260619 02:41:33.954030 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:46077
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:41:34.033672 16924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.033881 16924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.033900 16924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.035012 16924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:34.035053 16924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.035066 16924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.035079 16924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:34.035090 16924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:34.036373 16924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:46077
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.036563 16924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.036829 16924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.041846 16932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.041846 16930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.041913 16924 server_base.cc:1034] running on GCE node
W20260619 02:41:34.041851 16929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.042332 16924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.042625 16924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.043815 16924 hybrid_clock.cc:648] HybridClock initialized: now 1781836894043802 us; error 49 us; skew 500 ppm
I20260619 02:41:34.045138 16924 webserver.cc:460] Webserver started at http://127.7.137.62:42397/ using document root <none> and password file <none>
I20260619 02:41:34.045394 16924 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.045442 16924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.045572 16924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.046674 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "1e4493d7bc564cef9c04978c8c5953ea"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "4eaf97eb5898f9edf19e409c8158c19c"
server_key_iv: "3a0bfe0ea5705cfce118ff9f449204a5"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.047029 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "1e4493d7bc564cef9c04978c8c5953ea"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "4eaf97eb5898f9edf19e409c8158c19c"
server_key_iv: "3a0bfe0ea5705cfce118ff9f449204a5"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.048298 16924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:41:34.049083 16938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.049261 16924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.049350 16924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "1e4493d7bc564cef9c04978c8c5953ea"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "4eaf97eb5898f9edf19e409c8158c19c"
server_key_iv: "3a0bfe0ea5705cfce118ff9f449204a5"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.049422 16924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.067881 16924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.068224 16924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.068388 16924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.072242 16924 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:46077
I20260619 02:41:34.072294 16990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:46077 every 8 connection(s)
I20260619 02:41:34.072679 16924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:34.073457 16991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:34.076099 16991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea: Bootstrap starting.
I20260619 02:41:34.076709 16991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:34.077095 16991 log.cc:824] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:34.077772 16991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea: No bootstrap required, opened a new log
I20260619 02:41:34.078150 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 16924
I20260619 02:41:34.078248 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:34.078452 7716 external_mini_cluster.cc:1428] Setting key 6485bdc172b2d3c7dbb46ab6ab72ebb6
I20260619 02:41:34.080025 16991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } }
I20260619 02:41:34.080163 16991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:34.080185 16991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4493d7bc564cef9c04978c8c5953ea, State: Initialized, Role: FOLLOWER
I20260619 02:41:34.080345 16991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } }
I20260619 02:41:34.080410 16991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:34.080431 16991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:34.080498 16991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:34.081190 16991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } }
I20260619 02:41:34.081296 16991 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4493d7bc564cef9c04978c8c5953ea; no voters:
I20260619 02:41:34.081486 16991 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:34.081521 16996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:34.081634 16996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [term 1 LEADER]: Becoming Leader. State: Replica: 1e4493d7bc564cef9c04978c8c5953ea, State: Running, Role: LEADER
I20260619 02:41:34.081753 16991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:34.081789 16996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } }
I20260619 02:41:34.082161 16996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4493d7bc564cef9c04978c8c5953ea. Latest consensus state: current_term: 1 leader_uuid: "1e4493d7bc564cef9c04978c8c5953ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } } }
I20260619 02:41:34.082173 16998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e4493d7bc564cef9c04978c8c5953ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4493d7bc564cef9c04978c8c5953ea" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 46077 } } }
I20260619 02:41:34.082345 16996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:34.082355 16998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:34.082603 17005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:34.083174 17005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:34.084960 17005 catalog_manager.cc:1347] Generated new cluster ID: ae4c65008bd54ac29c197e9a11ffddcd
I20260619 02:41:34.085022 17005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:34.085192 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.078991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.7.137.62:46077)
0619 02:41:34.079216 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.079224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.079257 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:41:34.079701 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.079708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.079767 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.079915 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.080109 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.081587 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.082662 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.083259 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.083377 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.084445 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.084448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.084448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.084659 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.084691 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.084701 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.084793 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.084872 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.084950 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260619 02:41:34.097043 17005 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:34.097702 17005 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:34.101753 17005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e4493d7bc564cef9c04978c8c5953ea: Generated new TSK 0
I20260619 02:41:34.101936 17005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:34.103396 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--builtin_ntp_servers=127.7.137.20:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:41:34.201752 17015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.201949 17015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.201970 17015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.203140 17015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.203183 17015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.203208 17015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:34.204523 17015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.204789 17015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.205112 17015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.211122 17021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.211373 17020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.211545 17023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.212026 17015 server_base.cc:1034] running on GCE node
I20260619 02:41:34.212224 17015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.212491 17015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.213604 17015 hybrid_clock.cc:648] HybridClock initialized: now 1781836894213578 us; error 41 us; skew 500 ppm
I20260619 02:41:34.215049 17015 webserver.cc:460] Webserver started at http://127.7.137.1:34891/ using document root <none> and password file <none>
I20260619 02:41:34.215337 17015 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.215396 17015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.215520 17015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.216917 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "9eae1473f56f4b30a8554825460a8410"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "45396a7abaa6b27a2674fb7334ef03ea"
server_key_iv: "6841d3cfd11275b3ddce7cda791915e2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.217343 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "9eae1473f56f4b30a8554825460a8410"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "45396a7abaa6b27a2674fb7334ef03ea"
server_key_iv: "6841d3cfd11275b3ddce7cda791915e2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.219022 17015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:34.219866 17029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.220063 17015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:34.220165 17015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "9eae1473f56f4b30a8554825460a8410"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "45396a7abaa6b27a2674fb7334ef03ea"
server_key_iv: "6841d3cfd11275b3ddce7cda791915e2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.220238 17015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.244061 17015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.244359 17015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.244465 17015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.244660 17015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:34.245021 17015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:34.245054 17015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.245076 17015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:34.245090 17015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.251188 17015 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:44505
I20260619 02:41:34.251256 17142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:44505 every 8 connection(s)
I20260619 02:41:34.251592 17015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:34.256686 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.252335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:59323 (local address 127.7.137.62:46077)
0619 02:41:34.252380 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.252383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.252739 (+ 356us) server_negotiation.cc:407] Connection header received
0619 02:41:34.253192 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.253193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.253203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.253261 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.253886 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.254384 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.255330 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.255536 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.255594 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.256439 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.256440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.256441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.256444 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.256458 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.256461 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.256508 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.256594 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.256625 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:34.256939 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.252345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:46077 (local address 127.7.137.1:59323)
0619 02:41:34.252531 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:34.252565 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:34.253103 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:34.253316 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:34.253319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:34.253328 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:34.253812 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.253815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.254466 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.254467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.255245 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.255247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.255613 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.255614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.255719 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.256306 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:34.256306 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:34.256335 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:34.256358 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:34.256522 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:34.256526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:34.256529 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:34.256582 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:41:34.256660 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260619 02:41:34.257529 17143 heartbeater.cc:347] Connected to a master server at 127.7.137.62:46077
I20260619 02:41:34.257638 17143 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:34.257828 17143 heartbeater.cc:511] Master 127.7.137.62:46077 requested a full tablet report, sending...
I20260619 02:41:34.258373 16955 ts_manager.cc:194] Registered new tserver with Master: 9eae1473f56f4b30a8554825460a8410 (127.7.137.1:44505)
I20260619 02:41:34.258955 16955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.1:59323
I20260619 02:41:34.259199 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17015
I20260619 02:41:34.259292 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:34.259521 7716 external_mini_cluster.cc:1428] Setting key 6f134050908c98500c5ed1591ec529c0
I20260619 02:41:34.260596 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--builtin_ntp_servers=127.7.137.20:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:41:34.340261 17146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.340494 17146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.340556 17146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.341774 17146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.341838 17146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.341881 17146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:34.343474 17146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.343739 17146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.344030 17146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.348953 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.348953 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.348968 17151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.348968 17146 server_base.cc:1034] running on GCE node
I20260619 02:41:34.349272 17146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.349493 17146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.350641 17146 hybrid_clock.cc:648] HybridClock initialized: now 1781836894350593 us; error 64 us; skew 500 ppm
I20260619 02:41:34.351886 17146 webserver.cc:460] Webserver started at http://127.7.137.2:37753/ using document root <none> and password file <none>
I20260619 02:41:34.352206 17146 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.352268 17146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.352460 17146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.353590 17146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "7b9cc3d50d7241768f44d74c4dee28f3"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "af7532df7bd819d0ddbbd08e00082086"
server_key_iv: "11a4e1d82395cfb79849c9252d3fb9c3"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.353940 17146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "7b9cc3d50d7241768f44d74c4dee28f3"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "af7532df7bd819d0ddbbd08e00082086"
server_key_iv: "11a4e1d82395cfb79849c9252d3fb9c3"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.355312 17146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:34.356207 17160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.356428 17146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:41:34.356527 17146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "7b9cc3d50d7241768f44d74c4dee28f3"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "af7532df7bd819d0ddbbd08e00082086"
server_key_iv: "11a4e1d82395cfb79849c9252d3fb9c3"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.356604 17146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.370357 17146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.370744 17146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.370915 17146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.371160 17146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:34.371517 17146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:34.371572 17146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.371609 17146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:34.371639 17146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.377349 17146 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:33161
I20260619 02:41:34.377408 17273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:33161 every 8 connection(s)
I20260619 02:41:34.377760 17146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:34.383175 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.378478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:55727 (local address 127.7.137.62:46077)
0619 02:41:34.378526 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.378528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.378812 (+ 284us) server_negotiation.cc:407] Connection header received
0619 02:41:34.379366 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.379367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.379379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.379431 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.380130 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.380641 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.381694 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.381950 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.382026 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.382832 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.382833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.382833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.382838 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.382853 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.382857 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.382909 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.383062 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.383095 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:41:34.383383 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.378488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:46077 (local address 127.7.137.2:55727)
0619 02:41:34.378689 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:34.378723 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:34.379245 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:34.379515 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:34.379524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:34.379532 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:34.380021 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.380026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.380761 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.380762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.381557 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.381560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.382036 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.382038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.382173 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.382661 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:34.382661 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:34.382691 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:34.382713 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:34.382976 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:34.382981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:34.382983 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:41:34.383065 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:41:34.383168 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260619 02:41:34.383917 17274 heartbeater.cc:347] Connected to a master server at 127.7.137.62:46077
I20260619 02:41:34.384016 17274 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:34.384198 17274 heartbeater.cc:511] Master 127.7.137.62:46077 requested a full tablet report, sending...
I20260619 02:41:34.384620 16955 ts_manager.cc:194] Registered new tserver with Master: 7b9cc3d50d7241768f44d74c4dee28f3 (127.7.137.2:33161)
I20260619 02:41:34.385001 16955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.2:55727
I20260619 02:41:34.385308 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17146
I20260619 02:41:34.385396 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:34.385598 7716 external_mini_cluster.cc:1428] Setting key 855f18f551f233faf791faa42a220aac
I20260619 02:41:34.386371 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--builtin_ntp_servers=127.7.137.20:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:41:34.466096 17277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.466324 17277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.466358 17277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.467470 17277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.467538 17277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.467582 17277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:34.468959 17277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.469220 17277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.469511 17277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.474502 17282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.474488 17283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.474484 17285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.474526 17277 server_base.cc:1034] running on GCE node
I20260619 02:41:34.474910 17277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.475097 17277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.476240 17277 hybrid_clock.cc:648] HybridClock initialized: now 1781836894476209 us; error 47 us; skew 500 ppm
I20260619 02:41:34.477423 17277 webserver.cc:460] Webserver started at http://127.7.137.3:44117/ using document root <none> and password file <none>
I20260619 02:41:34.477653 17277 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.477697 17277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.477849 17277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.478840 17277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "5b2f7958329f4d68a71c4da3ab7ac6e5"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "f332ecfbd5b9845fbede8dae32b7aaa0"
server_key_iv: "7a6954a527646f065b4ba202151818c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.479183 17277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "5b2f7958329f4d68a71c4da3ab7ac6e5"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "f332ecfbd5b9845fbede8dae32b7aaa0"
server_key_iv: "7a6954a527646f065b4ba202151818c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.480422 17277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:34.481274 17291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.481446 17277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.481541 17277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "5b2f7958329f4d68a71c4da3ab7ac6e5"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "f332ecfbd5b9845fbede8dae32b7aaa0"
server_key_iv: "7a6954a527646f065b4ba202151818c4"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.481632 17277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.497287 17277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.497607 17277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.497758 17277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.497991 17277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:34.498332 17277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:34.498395 17277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.498438 17277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:34.498468 17277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.504715 17277 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:39423
I20260619 02:41:34.504776 17404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:39423 every 8 connection(s)
I20260619 02:41:34.505101 17277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:34.509907 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.505832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:41423 (local address 127.7.137.62:46077)
0619 02:41:34.505911 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.505913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.506071 (+ 158us) server_negotiation.cc:407] Connection header received
0619 02:41:34.506515 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.506516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.506525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.506571 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.507219 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.507699 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.508647 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.508901 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.508956 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.509683 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.509685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.509685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.509689 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.509702 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.509705 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.509763 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.509828 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.509850 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:41:34.510125 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.505754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:46077 (local address 127.7.137.3:41423)
0619 02:41:34.505926 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:34.505965 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:34.506435 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:34.506634 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:34.506637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:34.506645 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:34.507142 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.507146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.507780 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.507781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.508575 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:34.508577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:34.508971 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:34.508972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:34.509085 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:34.509545 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:34.509546 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:34.509573 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:34.509593 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:34.509765 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:34.509768 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:34.509770 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:41:34.509814 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:41:34.509886 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260619 02:41:34.510667 17405 heartbeater.cc:347] Connected to a master server at 127.7.137.62:46077
I20260619 02:41:34.510764 17405 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:34.510759 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17277
I20260619 02:41:34.510833 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:34.510972 17405 heartbeater.cc:511] Master 127.7.137.62:46077 requested a full tablet report, sending...
I20260619 02:41:34.511078 7716 external_mini_cluster.cc:1428] Setting key d918c6d1ff93ae7594f4a784189d808a
I20260619 02:41:34.511360 16955 ts_manager.cc:194] Registered new tserver with Master: 5b2f7958329f4d68a71c4da3ab7ac6e5 (127.7.137.3:39423)
I20260619 02:41:34.511847 16955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.3:41423
I20260619 02:41:34.511919 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:41:34.513675 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:34.513200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.7.137.62:46077)
0619 02:41:34.513233 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.513235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.513377 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:41:34.513447 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.513448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.513456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.513506 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.513620 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51268: BlockingRecv error: recv got EOF from 127.0.0.1:51268 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:41:34.513756 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17015
I20260619 02:41:34.519097 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17146
I20260619 02:41:34.523366 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17277
I20260619 02:41:34.527217 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 16924
2026-06-19T02:41:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (599 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T02:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:34Z Disabled control of system clock
I20260619 02:41:34.553009 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36915
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:41:34.632428 17416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.632627 17416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.632648 17416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.633780 17416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:34.633821 17416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.633834 17416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.633847 17416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:34.633859 17416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:34.635114 17416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36915
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.635293 17416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.635519 17416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.640219 17422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.640254 17416 server_base.cc:1034] running on GCE node
W20260619 02:41:34.640239 17424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.640239 17421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.640612 17416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.640897 17416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.642031 17416 hybrid_clock.cc:648] HybridClock initialized: now 1781836894642021 us; error 31 us; skew 500 ppm
I20260619 02:41:34.643169 17416 webserver.cc:460] Webserver started at http://127.7.137.62:46841/ using document root <none> and password file <none>
I20260619 02:41:34.643363 17416 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.643402 17416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.643517 17416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.644466 17416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "a30252a0a16d46319b1144ed79e7fb2c"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "b90ae80e3c3445dcea5f5831f729d00d"
server_key_iv: "5036ec974cca62b24bf1e7ad94bf0346"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.644821 17416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "a30252a0a16d46319b1144ed79e7fb2c"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "b90ae80e3c3445dcea5f5831f729d00d"
server_key_iv: "5036ec974cca62b24bf1e7ad94bf0346"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.646009 17416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:34.646687 17430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.646848 17416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:34.646948 17416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "a30252a0a16d46319b1144ed79e7fb2c"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "b90ae80e3c3445dcea5f5831f729d00d"
server_key_iv: "5036ec974cca62b24bf1e7ad94bf0346"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.647028 17416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.650290 17416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.650543 17416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.650689 17416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.654229 17416 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36915
I20260619 02:41:34.654259 17482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36915 every 8 connection(s)
I20260619 02:41:34.654573 17416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:34.655227 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:34.656823 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17416
I20260619 02:41:34.656893 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:34.657086 7716 external_mini_cluster.cc:1428] Setting key 9320c224161e6ff6c075721bdd03fa27
I20260619 02:41:34.658401 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c: Bootstrap starting.
I20260619 02:41:34.659070 17483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:34.659371 17483 log.cc:824] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:34.659637 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.657626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.7.137.62:36915)
0619 02:41:34.657822 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.657831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.657866 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:41:34.658367 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.658376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.658427 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.658583 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.659217 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.659220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.659221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.659225 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.659270 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.659279 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.659368 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.659409 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.659465 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260619 02:41:34.660107 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c: No bootstrap required, opened a new log
I20260619 02:41:34.661507 17483 raft_consensus.cc:348] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } }
I20260619 02:41:34.661633 17483 raft_consensus.cc:374] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:34.661655 17483 raft_consensus.cc:729] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30252a0a16d46319b1144ed79e7fb2c, State: Initialized, Role: FOLLOWER
I20260619 02:41:34.661790 17483 consensus_queue.cc:260] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } }
I20260619 02:41:34.661870 17483 raft_consensus.cc:388] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:34.661912 17483 raft_consensus.cc:482] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:34.661958 17483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:34.662736 17483 raft_consensus.cc:504] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } }
I20260619 02:41:34.662858 17483 leader_election.cc:302] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a30252a0a16d46319b1144ed79e7fb2c; no voters:
I20260619 02:41:34.663029 17483 leader_election.cc:288] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:34.663064 17488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:34.663224 17488 raft_consensus.cc:686] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [term 1 LEADER]: Becoming Leader. State: Replica: a30252a0a16d46319b1144ed79e7fb2c, State: Running, Role: LEADER
I20260619 02:41:34.663300 17483 sys_catalog.cc:564] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:34.663309 17488 consensus_queue.cc:237] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } }
I20260619 02:41:34.663669 17489 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a30252a0a16d46319b1144ed79e7fb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } } }
I20260619 02:41:34.663767 17489 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:34.663964 17494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:34.664065 17490 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a30252a0a16d46319b1144ed79e7fb2c. Latest consensus state: current_term: 1 leader_uuid: "a30252a0a16d46319b1144ed79e7fb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30252a0a16d46319b1144ed79e7fb2c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36915 } } }
I20260619 02:41:34.664184 17490 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:34.664422 17494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:41:34.665899 17494 catalog_manager.cc:1347] Generated new cluster ID: be2143f429ae4df4b001bb2d8cdd7739
I20260619 02:41:34.665951 17494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:34.687356 17494 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:34.687557 17494 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:34.692505 17494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a30252a0a16d46319b1144ed79e7fb2c: Generated new TSK 0
I20260619 02:41:34.692689 17494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:41:34.700814 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--builtin_ntp_servers=127.7.137.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:41:34.783354 17507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.783548 17507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.783571 17507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.784694 17507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.784758 17507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.784807 17507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:34.786103 17507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.786329 17507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.786628 17507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.791823 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.791867 17507 server_base.cc:1034] running on GCE node
W20260619 02:41:34.791838 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.791949 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.792205 17507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.792393 17507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.793526 17507 hybrid_clock.cc:648] HybridClock initialized: now 1781836894793507 us; error 41 us; skew 500 ppm
I20260619 02:41:34.794750 17507 webserver.cc:460] Webserver started at http://127.7.137.1:38681/ using document root <none> and password file <none>
I20260619 02:41:34.794941 17507 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.794981 17507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.795065 17507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.796093 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "fffa8a3e0ba446289ed0cbf3508029e1"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "29c7b5c578f15c9c4434989513bbb9ee"
server_key_iv: "9756a6a679e8288f7853ad7da1657439"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.796377 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "fffa8a3e0ba446289ed0cbf3508029e1"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "29c7b5c578f15c9c4434989513bbb9ee"
server_key_iv: "9756a6a679e8288f7853ad7da1657439"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.797704 17507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:34.798398 17521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.798619 17507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:34.798717 17507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "fffa8a3e0ba446289ed0cbf3508029e1"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "29c7b5c578f15c9c4434989513bbb9ee"
server_key_iv: "9756a6a679e8288f7853ad7da1657439"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.798801 17507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.802232 17507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.802481 17507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.802632 17507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.802830 17507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:34.803112 17507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:34.803162 17507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.803200 17507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:34.803226 17507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.809002 17507 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:33197
I20260619 02:41:34.809044 17634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:33197 every 8 connection(s)
I20260619 02:41:34.809357 17507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:34.812532 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.810060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:58347 (local address 127.7.137.62:36915)
0619 02:41:34.810118 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.810120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.810334 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:41:34.810774 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.810774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.810779 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.810837 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.812152 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.812154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.812154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.812159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.812190 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.812195 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.812249 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.812404 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.812421 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:41:34.812659 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.810012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36915 (local address 127.7.137.1:58347)
0619 02:41:34.810202 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:34.810244 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:34.810697 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:34.810886 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:34.810889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:34.810897 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:34.812004 (+ 1107us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:34.812005 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:34.812043 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:34.812080 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:34.812289 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:34.812294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:34.812297 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:34.812367 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:41:34.812428 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260619 02:41:34.813344 17635 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36915
I20260619 02:41:34.813441 17635 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:34.813583 17635 heartbeater.cc:511] Master 127.7.137.62:36915 requested a full tablet report, sending...
I20260619 02:41:34.814061 17447 ts_manager.cc:194] Registered new tserver with Master: fffa8a3e0ba446289ed0cbf3508029e1 (127.7.137.1:33197)
I20260619 02:41:34.815543 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17507
I20260619 02:41:34.815640 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:34.815822 7716 external_mini_cluster.cc:1428] Setting key 03ed9fef52db76b66e1eb2bf399193c4
I20260619 02:41:34.816673 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--builtin_ntp_servers=127.7.137.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:41:34.895925 17638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:34.896121 17638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:34.896144 17638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:34.897297 17638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:34.897336 17638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:34.897360 17638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:34.898651 17638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:34.898881 17638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:34.899163 17638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:34.904075 17643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.904153 17638 server_base.cc:1034] running on GCE node
W20260619 02:41:34.904134 17644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:34.904124 17646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:34.904505 17638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:34.904789 17638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:34.905984 17638 hybrid_clock.cc:648] HybridClock initialized: now 1781836894905971 us; error 25 us; skew 500 ppm
I20260619 02:41:34.907238 17638 webserver.cc:460] Webserver started at http://127.7.137.2:44373/ using document root <none> and password file <none>
I20260619 02:41:34.907483 17638 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:34.907562 17638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:34.907711 17638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:34.908826 17638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "9b453d6b8dc34ba4811123818ac86013"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "1ee63e26596181d149c3c22735a3c890"
server_key_iv: "14c4a3fbc9caf939d89f6b4d90d139f2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.909183 17638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "9b453d6b8dc34ba4811123818ac86013"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "1ee63e26596181d149c3c22735a3c890"
server_key_iv: "14c4a3fbc9caf939d89f6b4d90d139f2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.910566 17638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:41:34.911299 17652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.911461 17638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:41:34.911554 17638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "9b453d6b8dc34ba4811123818ac86013"
format_stamp: "Formatted at 2026-06-19 02:41:34 on dist-test-slave-pdnb"
server_key: "1ee63e26596181d149c3c22735a3c890"
server_key_iv: "14c4a3fbc9caf939d89f6b4d90d139f2"
server_key_version: "encryptionkey@0"
I20260619 02:41:34.911633 17638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:34.914878 17638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:34.915117 17638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:34.915252 17638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:34.915447 17638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:34.915755 17638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:34.915808 17638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.915845 17638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:34.915874 17638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:34.921347 17638 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:37495
I20260619 02:41:34.921419 17765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:37495 every 8 connection(s)
I20260619 02:41:34.921725 17638 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:41:34.924655 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.922417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:35431 (local address 127.7.137.62:36915)
0619 02:41:34.922457 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:34.922459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:34.922695 (+ 236us) server_negotiation.cc:407] Connection header received
0619 02:41:34.923117 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:34.923118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:34.923122 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:34.923177 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:34.924379 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:34.924380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:34.924380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:34.924385 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:34.924405 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:34.924409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:34.924465 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:34.924587 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 02:41:34.924602 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:41:34.924883 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:34.922380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36915 (local address 127.7.137.2:35431)
0619 02:41:34.922566 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:34.922604 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:34.923036 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:34.923250 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:34.923253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:34.923261 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:34.924223 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:34.924223 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:34.924249 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:34.924276 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:34.924496 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:34.924501 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:34.924504 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:41:34.924586 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:41:34.924634 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260619 02:41:34.925346 17766 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36915
I20260619 02:41:34.925448 17766 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:34.925593 17766 heartbeater.cc:511] Master 127.7.137.62:36915 requested a full tablet report, sending...
I20260619 02:41:34.925964 17447 ts_manager.cc:194] Registered new tserver with Master: 9b453d6b8dc34ba4811123818ac86013 (127.7.137.2:37495)
I20260619 02:41:34.931094 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17638
I20260619 02:41:34.931174 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:34.931385 7716 external_mini_cluster.cc:1428] Setting key 34cc140c734babfb63e9e80d1f89e2ba
I20260619 02:41:34.932277 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--builtin_ntp_servers=127.7.137.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:41:35.012603 17769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:35.012847 17769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:35.012882 17769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:35.014109 17769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:35.014161 17769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:35.014204 17769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:35.015650 17769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:35.015902 17769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:35.016193 17769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:35.021164 17775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:35.021176 17774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:35.021180 17777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:35.021507 17769 server_base.cc:1034] running on GCE node
I20260619 02:41:35.021741 17769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:35.022002 17769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:35.023164 17769 hybrid_clock.cc:648] HybridClock initialized: now 1781836895023153 us; error 29 us; skew 500 ppm
I20260619 02:41:35.024439 17769 webserver.cc:460] Webserver started at http://127.7.137.3:42417/ using document root <none> and password file <none>
I20260619 02:41:35.024698 17769 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:35.024801 17769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:35.024979 17769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:35.026008 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "7d2e3849b2a841f9ac85952b7ab38496"
format_stamp: "Formatted at 2026-06-19 02:41:35 on dist-test-slave-pdnb"
server_key: "e3a0fa6e8bbd41d24af164bf0b17e84d"
server_key_iv: "cc61a6908f83d9eb4fdbd3a80f637754"
server_key_version: "encryptionkey@0"
I20260619 02:41:35.026364 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "7d2e3849b2a841f9ac85952b7ab38496"
format_stamp: "Formatted at 2026-06-19 02:41:35 on dist-test-slave-pdnb"
server_key: "e3a0fa6e8bbd41d24af164bf0b17e84d"
server_key_iv: "cc61a6908f83d9eb4fdbd3a80f637754"
server_key_version: "encryptionkey@0"
I20260619 02:41:35.027611 17769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:35.028342 17783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:35.028541 17769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:41:35.028609 17769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "7d2e3849b2a841f9ac85952b7ab38496"
format_stamp: "Formatted at 2026-06-19 02:41:35 on dist-test-slave-pdnb"
server_key: "e3a0fa6e8bbd41d24af164bf0b17e84d"
server_key_iv: "cc61a6908f83d9eb4fdbd3a80f637754"
server_key_version: "encryptionkey@0"
I20260619 02:41:35.028693 17769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:35.031922 17769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:35.032187 17769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:35.032317 17769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:35.032521 17769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:35.032878 17769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:35.032930 17769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:35.032970 17769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:35.032996 17769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:35.038630 17769 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:33927
I20260619 02:41:35.038671 17896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:33927 every 8 connection(s)
I20260619 02:41:35.038975 17769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:41:35.041913 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:35.039627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:49773 (local address 127.7.137.62:36915)
0619 02:41:35.039671 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:35.039673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:35.039922 (+ 249us) server_negotiation.cc:407] Connection header received
0619 02:41:35.040421 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:35.040422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:35.040428 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:35.040478 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:35.041667 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:35.041668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:35.041669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:41:35.041673 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:35.041688 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:41:35.041691 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:35.041743 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:35.041830 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:41:35.041845 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:41:35.042105 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:35.039632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36915 (local address 127.7.137.3:49773)
0619 02:41:35.039812 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:35.039843 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:35.040306 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:35.040555 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:35.040559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:35.040568 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:35.041517 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:41:35.041517 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:35.041544 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:41:35.041570 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:35.041740 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:35.041744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:35.041748 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:35.041840 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:41:35.041898 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260619 02:41:35.042583 17897 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36915
I20260619 02:41:35.042676 17897 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:35.042812 17897 heartbeater.cc:511] Master 127.7.137.62:36915 requested a full tablet report, sending...
I20260619 02:41:35.043170 17447 ts_manager.cc:194] Registered new tserver with Master: 7d2e3849b2a841f9ac85952b7ab38496 (127.7.137.3:33927)
I20260619 02:41:35.046649 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17769
I20260619 02:41:35.046720 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:35.046866 7716 external_mini_cluster.cc:1428] Setting key c98ad044a1976bf860db4e95213dc267
I20260619 02:41:35.047690 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:41:35.049348 17486 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:35.048934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.7.137.62:36915)
0619 02:41:35.048971 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:35.048973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:35.049091 (+ 118us) server_negotiation.cc:407] Connection header received
0619 02:41:35.049127 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:35.049128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:35.049131 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:35.049174 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:35.049287 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44688: BlockingRecv error: recv got EOF from 127.0.0.1:44688 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:41:35.049415 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17507
I20260619 02:41:35.053947 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17638
I20260619 02:41:35.057850 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17769
I20260619 02:41:35.061765 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17416
2026-06-19T02:41:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (534 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:35 dist-test-slave-pdnb krb5kdc[17907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:35 dist-test-slave-pdnb krb5kdc[17907](info): set up 2 sockets
Jun 19 02:41:35 dist-test-slave-pdnb krb5kdc[17907](info): commencing operation
krb5kdc: starting...
W20260619 02:41:36.089627 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:41:36.200855 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42669
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:36.279882 17920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:36.280078 17920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:36.280102 17920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:36.281292 17920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:41:36.281334 17920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:36.281348 17920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:36.281361 17920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:41:36.281373 17920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:41:36.282791 17920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:42669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:36.282965 17920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:36.283119 17920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:36.287866 17926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.287875 17925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.287884 17928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:36.287949 17920 server_base.cc:1034] running on GCE node
I20260619 02:41:36.288328 17920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:36.288600 17920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:36.289763 17920 hybrid_clock.cc:648] HybridClock initialized: now 1781836896289739 us; error 35 us; skew 500 ppm
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:36.292024 17920 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:41:36.292466 17920 webserver.cc:460] Webserver started at http://127.7.137.62:40573/ using document root <none> and password file <none>
I20260619 02:41:36.292675 17920 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:36.292755 17920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:36.292893 17920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:36.293905 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "dc4a82b6673049bd9809fc0a8937f678"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "87a6c41cb96cc5bfde4c7da1e839eab7"
server_key_iv: "6bb575cdf49194d7909460534be0e1b9"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.294273 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "dc4a82b6673049bd9809fc0a8937f678"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "87a6c41cb96cc5bfde4c7da1e839eab7"
server_key_iv: "6bb575cdf49194d7909460534be0e1b9"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.295596 17920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:36.296277 17935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.296442 17920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:36.296535 17920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "dc4a82b6673049bd9809fc0a8937f678"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "87a6c41cb96cc5bfde4c7da1e839eab7"
server_key_iv: "6bb575cdf49194d7909460534be0e1b9"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.296617 17920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:36.324375 17920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:36.326751 17920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:36.326915 17920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:36.330925 17920 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:42669
I20260619 02:41:36.330963 17987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:42669 every 8 connection(s)
I20260619 02:41:36.331341 17920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:41:36.332285 17988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:36.334554 17988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: Bootstrap starting.
I20260619 02:41:36.335143 17988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:36.335201 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 17920
I20260619 02:41:36.335296 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:41:36.335485 7716 external_mini_cluster.cc:1428] Setting key ad8cee369346ef95f466578bc213c09d
I20260619 02:41:36.335497 17988 log.cc:824] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:36.336304 17988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: No bootstrap required, opened a new log
I20260619 02:41:36.338761 17988 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } }
I20260619 02:41:36.338937 17988 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:36.338971 17988 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4a82b6673049bd9809fc0a8937f678, State: Initialized, Role: FOLLOWER
I20260619 02:41:36.339095 17988 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } }
I20260619 02:41:36.339174 17988 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:41:36.339200 17988 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:41:36.339244 17988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:36.339867 17988 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } }
I20260619 02:41:36.339960 17988 leader_election.cc:302] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc4a82b6673049bd9809fc0a8937f678; no voters:
I20260619 02:41:36.340140 17988 leader_election.cc:288] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:41:36.340181 17993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:36.340308 17993 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [term 1 LEADER]: Becoming Leader. State: Replica: dc4a82b6673049bd9809fc0a8937f678, State: Running, Role: LEADER
I20260619 02:41:36.340405 17988 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:41:36.340708 17993 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } }
I20260619 02:41:36.341144 17994 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc4a82b6673049bd9809fc0a8937f678. Latest consensus state: current_term: 1 leader_uuid: "dc4a82b6673049bd9809fc0a8937f678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } } }
I20260619 02:41:36.341231 17994 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:36.341234 17995 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc4a82b6673049bd9809fc0a8937f678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4a82b6673049bd9809fc0a8937f678" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42669 } } }
I20260619 02:41:36.341311 17995 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678 [sys.catalog]: This master's current role is: LEADER
I20260619 02:41:36.341432 18002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:41:36.342131 18002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:36.343641 18002 catalog_manager.cc:1347] Generated new cluster ID: 9f9cbcc7c6274356a8d832704988eb03
I20260619 02:41:36.343691 18002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:41:36.346701 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.336010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.7.137.62:42669)
0619 02:41:36.336194 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.336201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.336226 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:41:36.336677 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.336684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.336774 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.336894 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:36.337648 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.338501 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.339530 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.339949 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.340079 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.343067 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:36.343072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:36.343072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:36.343084 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:36.345120 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.345487 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.345490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.345491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.345596 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.345801 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.345802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.345802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.346100 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:36.346246 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.346355 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.346490 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
I20260619 02:41:36.348949 18002 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:41:36.349675 18002 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:41:36.353139 18002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: Generated new TSK 0
I20260619 02:41:36.353298 18002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:41:36.380986 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:36.461169 18014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:36.461369 18014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:36.461390 18014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:36.462536 18014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:36.462579 18014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:36.462612 18014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:41:36.464090 18014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:36.464351 18014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:36.464579 18014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:36.469506 18020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.469477 18019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.469501 18022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:36.469635 18014 server_base.cc:1034] running on GCE node
I20260619 02:41:36.469950 18014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:36.470139 18014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:36.471271 18014 hybrid_clock.cc:648] HybridClock initialized: now 1781836896471252 us; error 35 us; skew 500 ppm
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:36.473718 18014 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:41:36.474125 18014 webserver.cc:460] Webserver started at http://127.7.137.1:42989/ using document root <none> and password file <none>
I20260619 02:41:36.474371 18014 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:36.474416 18014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:36.474557 18014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:36.475575 18014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "3cd9e84e5aae4ee3e5a03bba0bf928c7"
server_key_iv: "654c05e2e11b0d40cad1a25d7a5d5b82"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.475935 18014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "3cd9e84e5aae4ee3e5a03bba0bf928c7"
server_key_iv: "654c05e2e11b0d40cad1a25d7a5d5b82"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.477283 18014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:41:36.477942 18029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.478117 18014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:36.478212 18014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "3cd9e84e5aae4ee3e5a03bba0bf928c7"
server_key_iv: "654c05e2e11b0d40cad1a25d7a5d5b82"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.478293 18014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:36.502897 18014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:36.504753 18014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:36.504918 18014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:36.505149 18014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:36.505455 18014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:36.505512 18014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.505548 18014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:36.505575 18014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.511121 18014 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:38307
I20260619 02:41:36.511195 18142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:38307 every 8 connection(s)
I20260619 02:41:36.511518 18014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:41:36.515945 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 18014
I20260619 02:41:36.516037 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:41:36.516245 7716 external_mini_cluster.cc:1428] Setting key 16f3c264708464c9cf8a119021d302ed
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:36.521044 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.512221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:55013 (local address 127.7.137.62:42669)
0619 02:41:36.512266 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.512268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.512473 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:41:36.512965 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.512966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.512975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.513024 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:36.513903 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.514398 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.515320 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.515508 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.515557 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.518143 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:36.518145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:36.518145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:36.518152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:36.519669 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.520033 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.520034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.520035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.520097 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.520314 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.520315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.520315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.520465 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:36.520550 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.520830 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.520926 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:41:36.521070 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.512215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42669 (local address 127.7.137.1:55013)
0619 02:41:36.512356 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.512390 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.512887 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.513076 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.513079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.513633 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:36.513830 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.513834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.514478 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.514478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.515247 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.515249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.515579 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.515580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.515672 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.516217 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:36.516225 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:36.518048 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:36.519750 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.519756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.519757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.519960 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.520161 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.520162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.520162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.520253 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.520543 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:36.520547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:36.520653 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.520797 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.520916 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260619 02:41:36.521620 18143 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42669
I20260619 02:41:36.521725 18143 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:36.521921 18143 heartbeater.cc:511] Master 127.7.137.62:42669 requested a full tablet report, sending...
I20260619 02:41:36.522410 17952 ts_manager.cc:194] Registered new tserver with Master: 4df38c27625b47d0b0b2ea6bdbf1caa7 (127.7.137.1:38307)
I20260619 02:41:36.523012 17952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:55013
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:41:36.539067 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:36.619896 18148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:36.620093 18148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:36.620117 18148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:36.621209 18148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:36.621248 18148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:36.621273 18148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:41:36.622602 18148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:36.622776 18148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:36.622989 18148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:36.627812 18154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.627799 18156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.627802 18153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:36.628006 18148 server_base.cc:1034] running on GCE node
I20260619 02:41:36.628203 18148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:36.628397 18148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:36.629555 18148 hybrid_clock.cc:648] HybridClock initialized: now 1781836896629546 us; error 40 us; skew 500 ppm
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:36.631920 18148 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:41:36.632340 18148 webserver.cc:460] Webserver started at http://127.7.137.2:38739/ using document root <none> and password file <none>
I20260619 02:41:36.632513 18148 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:36.632551 18148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:36.632637 18148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:36.633708 18148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "a0b3cd033089403db6efc2075caf06b9"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "2bdbb8c92f4e63c8ab20dbdfa7f05526"
server_key_iv: "d1890b71373a3ba1bad0a30f65f9af8f"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.634016 18148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "a0b3cd033089403db6efc2075caf06b9"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "2bdbb8c92f4e63c8ab20dbdfa7f05526"
server_key_iv: "d1890b71373a3ba1bad0a30f65f9af8f"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.635221 18148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:36.635879 18163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.636029 18148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:36.636108 18148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "a0b3cd033089403db6efc2075caf06b9"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "2bdbb8c92f4e63c8ab20dbdfa7f05526"
server_key_iv: "d1890b71373a3ba1bad0a30f65f9af8f"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.636159 18148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:36.647797 18148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:36.650352 18148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:36.650462 18148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:36.650655 18148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:36.650966 18148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:36.651007 18148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.651031 18148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:36.651043 18148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.656648 18148 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:37667
I20260619 02:41:36.656792 18276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:37667 every 8 connection(s)
I20260619 02:41:36.657095 18148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:36.663626 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 18148
I20260619 02:41:36.663700 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:41:36.663885 7716 external_mini_cluster.cc:1428] Setting key 01f192e3056449e2810af1f58dda7f0c
I20260619 02:41:36.666739 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.657818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42669 (local address 127.7.137.2:55189)
0619 02:41:36.657982 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.658016 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.658520 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.658789 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.658793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.659380 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:36.659567 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.659572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.660267 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.660268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.661153 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.661155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.661542 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.661543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.661656 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.662147 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:36.662155 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:36.663448 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:36.665259 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.665265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.665267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.665585 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.665778 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.665779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.665779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.665918 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.666272 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:36.666277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:36.666388 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.666467 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.666582 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260619 02:41:36.666743 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.657837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:55189 (local address 127.7.137.62:42669)
0619 02:41:36.657902 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.657905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.658101 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:41:36.658643 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.658645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.658657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.658708 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:36.659671 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.660169 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.661243 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.661432 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.661522 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.663558 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:36.663559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:36.663559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:36.663566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:36.665159 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.665666 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.665667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.665668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.665719 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.666006 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.666008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.666008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.666171 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:36.666219 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.666531 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.666624 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:41:36.667301 18277 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42669
I20260619 02:41:36.667411 18277 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:36.667590 18277 heartbeater.cc:511] Master 127.7.137.62:42669 requested a full tablet report, sending...
I20260619 02:41:36.667994 17952 ts_manager.cc:194] Registered new tserver with Master: a0b3cd033089403db6efc2075caf06b9 (127.7.137.2:37667)
I20260619 02:41:36.668376 17952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:55189
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:41:36.688954 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:41:36.771250 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:41:36.771479 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:41:36.771519 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:41:36.772687 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:41:36.772775 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:41:36.772822 18282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:41:36.774272 18282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:41:36.774533 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:41:36.774767 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:41:36.779681 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:36.779806 18282 server_base.cc:1034] running on GCE node
W20260619 02:41:36.779698 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:41:36.779690 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:41:36.780145 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:41:36.780386 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:41:36.781549 18282 hybrid_clock.cc:648] HybridClock initialized: now 1781836896781554 us; error 43 us; skew 500 ppm
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:41:36.783880 18282 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:41:36.784314 18282 webserver.cc:460] Webserver started at http://127.7.137.3:42381/ using document root <none> and password file <none>
I20260619 02:41:36.784487 18282 fs_manager.cc:359] Metadata directory not provided
I20260619 02:41:36.784526 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:41:36.784612 18282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:41:36.785638 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "65e989542c614c65877ce11e8edc84cb"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "8a7332c3f21095e2bcfc9021ce216f2c"
server_key_iv: "074f44094eb41cf11fa882c836c1648b"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.785925 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "65e989542c614c65877ce11e8edc84cb"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "8a7332c3f21095e2bcfc9021ce216f2c"
server_key_iv: "074f44094eb41cf11fa882c836c1648b"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.787154 18282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:41:36.787847 18297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.788015 18282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:41:36.788094 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "65e989542c614c65877ce11e8edc84cb"
format_stamp: "Formatted at 2026-06-19 02:41:36 on dist-test-slave-pdnb"
server_key: "8a7332c3f21095e2bcfc9021ce216f2c"
server_key_iv: "074f44094eb41cf11fa882c836c1648b"
server_key_version: "encryptionkey@0"
I20260619 02:41:36.788146 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:41:36.797060 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:41:36.799360 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:41:36.799464 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:41:36.799636 18282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:41:36.799942 18282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:41:36.799995 18282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.800027 18282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:41:36.800056 18282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:41:36.805790 18282 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:42607
I20260619 02:41:36.805877 18410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:42607 every 8 connection(s)
I20260619 02:41:36.806169 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:41:36 dist-test-slave-pdnb krb5kdc[17907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:41:36.813170 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 18282
I20260619 02:41:36.813290 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:41:36.813513 7716 external_mini_cluster.cc:1428] Setting key a05918e9d83abfc896d6ba0be40b4506
I20260619 02:41:36.815141 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.806828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:49013 (local address 127.7.137.62:42669)
0619 02:41:36.806868 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.806870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.807065 (+ 195us) server_negotiation.cc:407] Connection header received
0619 02:41:36.807508 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.807510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.807521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.807570 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:36.808458 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.809070 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.810013 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.810206 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.810256 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.812362 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:36.812364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:36.812364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:36.812370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:36.813955 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.814382 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.814383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.814383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.814444 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.814685 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.814685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.814685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.814796 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:36.814847 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.815025 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.815080 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:41:36.815392 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.806803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42669 (local address 127.7.137.3:49013)
0619 02:41:36.806942 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.806975 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.807433 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.807621 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.807625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.808163 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:41:36.808357 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.808361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.809147 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.809148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.809936 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.809938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.810274 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.810275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.810370 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.810897 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:41:36.810905 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:41:36.812281 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:41:36.814050 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.814075 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.814077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.814321 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.814524 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:41:36.814525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:41:36.814527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:41:36.814629 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:41:36.814837 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:41:36.814840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:41:36.814942 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.815016 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.815132 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260619 02:41:36.815812 18411 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42669
I20260619 02:41:36.815914 18411 heartbeater.cc:464] Registering TS with master...
I20260619 02:41:36.816089 18411 heartbeater.cc:511] Master 127.7.137.62:42669 requested a full tablet report, sending...
I20260619 02:41:36.816465 17952 ts_manager.cc:194] Registered new tserver with Master: 65e989542c614c65877ce11e8edc84cb (127.7.137.3:42607)
I20260619 02:41:36.816934 17952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:49013
I20260619 02:41:36.825206 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:41:36.831995 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.826510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.7.137.62:42669)
0619 02:41:36.826617 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.826619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.826646 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:41:36.826707 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.826708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.826716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.826754 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:41:36.827271 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.827754 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.828475 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.828675 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.828778 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.829876 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:41:36.829877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:41:36.829877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:41:36.829883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:41:36.831034 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.831331 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.831332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.831332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.831377 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:41:36.831529 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:41:36.831530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:41:36.831530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:41:36.831678 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:41:36.831749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.831833 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.831891 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 02:41:36.833657 17952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:41:36.835225 17952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:41:36.841619 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:37667 (local address 127.0.0.1:50468)
0619 02:41:36.837522 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.837568 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.837630 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.838044 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.838047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.838049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:36.838115 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.838117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839104 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.839105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.840375 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.840378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.841337 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.841338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.841409 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.841424 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.841489 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.841541 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":55,"threads_started":1}
I20260619 02:41:36.841742 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38307 (local address 127.0.0.1:57758)
0619 02:41:36.837734 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.837744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.837826 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.838129 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.838130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.838132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:36.838184 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.838185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839488 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.839489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.840512 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.840514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.841489 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.841490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.841560 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.841564 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.841626 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.841664 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":39,"spinlock_wait_cycles":384,"thread_start_us":56,"threads_started":1}
I20260619 02:41:36.841913 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50468 (local address 127.7.137.2:37667)
0619 02:41:36.837768 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.837773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.837783 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:41:36.837827 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.837829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.837846 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.837922 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:36.838211 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839005 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.840456 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.841268 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.841339 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.841394 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.841802 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.841839 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":80,"threads_started":1}
I20260619 02:41:36.842231 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.7.137.1:38307)
0619 02:41:36.837731 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.837735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.837796 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:41:36.837925 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.837928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.837945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.838001 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:36.838271 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839370 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.840592 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.841403 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.841479 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.841535 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.841651 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.841985 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260619 02:41:36.842653 18211 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5731f86b3445db8771651cdbae9213 (DEFAULT_TABLE table=test-table [id=7c7ba2e310d24485a86b239948716255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:36.842854 18077 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5731f86b3445db8771651cdbae9213 (DEFAULT_TABLE table=test-table [id=7c7ba2e310d24485a86b239948716255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:36.842957 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5731f86b3445db8771651cdbae9213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:36.843168 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5731f86b3445db8771651cdbae9213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:36.845065 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42607 (local address 127.0.0.1:35724)
0619 02:41:36.837792 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.837803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.837856 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.838179 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.838179 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.838181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:36.838212 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.838213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839491 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.839492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.840642 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.840644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.844826 (+ 4182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.844827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.844893 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.844897 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.844963 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.845000 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":25,"thread_start_us":51,"threads_started":1}
I20260619 02:41:36.845419 18424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.837785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.7.137.3:42607)
0619 02:41:36.837956 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.837961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.837973 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:41:36.838021 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.838023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.838036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.838106 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:36.838287 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.839393 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.840745 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.841565 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.841634 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.841682 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.844992 (+ 3310us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.845038 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260619 02:41:36.845696 18425 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Bootstrap starting.
I20260619 02:41:36.845883 18426 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Bootstrap starting.
I20260619 02:41:36.846083 18345 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5731f86b3445db8771651cdbae9213 (DEFAULT_TABLE table=test-table [id=7c7ba2e310d24485a86b239948716255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:41:36.846405 18345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5731f86b3445db8771651cdbae9213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:41:36.846429 18425 tablet_bootstrap.cc:654] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:36.846493 18426 tablet_bootstrap.cc:654] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:36.846983 18425 log.cc:824] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:36.847030 18426 log.cc:824] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:36.847649 18425 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: No bootstrap required, opened a new log
I20260619 02:41:36.847748 18425 ts_tablet_manager.cc:1397] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:36.849278 18429 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Bootstrap starting.
I20260619 02:41:36.849319 18425 raft_consensus.cc:348] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.849538 18425 raft_consensus.cc:374] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:36.849588 18425 raft_consensus.cc:729] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b3cd033089403db6efc2075caf06b9, State: Initialized, Role: FOLLOWER
I20260619 02:41:36.849702 18425 consensus_queue.cc:260] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.849753 18426 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: No bootstrap required, opened a new log
I20260619 02:41:36.849809 18426 ts_tablet_manager.cc:1397] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 02:41:36.849954 18429 tablet_bootstrap.cc:654] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Neither blocks nor log segments found. Creating new log.
I20260619 02:41:36.849982 18425 ts_tablet_manager.cc:1428] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:36.850116 18277 heartbeater.cc:503] Master 127.7.137.62:42669 was elected leader, sending a full tablet report...
I20260619 02:41:36.850471 18429 log.cc:824] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:41:36.851233 18429 tablet_bootstrap.cc:492] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: No bootstrap required, opened a new log
I20260619 02:41:36.851305 18429 ts_tablet_manager.cc:1397] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:41:36.851457 18426 raft_consensus.cc:348] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.851605 18426 raft_consensus.cc:374] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:36.851648 18426 raft_consensus.cc:729] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df38c27625b47d0b0b2ea6bdbf1caa7, State: Initialized, Role: FOLLOWER
I20260619 02:41:36.851764 18426 consensus_queue.cc:260] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.852015 18426 ts_tablet_manager.cc:1428] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:41:36.852031 18143 heartbeater.cc:503] Master 127.7.137.62:42669 was elected leader, sending a full tablet report...
I20260619 02:41:36.853149 18429 raft_consensus.cc:348] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.853268 18429 raft_consensus.cc:374] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:41:36.853289 18429 raft_consensus.cc:729] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e989542c614c65877ce11e8edc84cb, State: Initialized, Role: FOLLOWER
I20260619 02:41:36.853387 18429 consensus_queue.cc:260] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.853694 18429 ts_tablet_manager.cc:1428] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:41:36.853705 18411 heartbeater.cc:503] Master 127.7.137.62:42669 was elected leader, sending a full tablet report...
W20260619 02:41:36.907691 18278 tablet.cc:2369] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:36.974334 18432 raft_consensus.cc:482] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:41:36.974460 18432 raft_consensus.cc:504] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.974805 18432 leader_election.cc:288] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0b3cd033089403db6efc2075caf06b9 (127.7.137.2:37667), 65e989542c614c65877ce11e8edc84cb (127.7.137.3:42607)
I20260619 02:41:36.978415 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.975116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:59221 (local address 127.7.137.2:37667)
0619 02:41:36.975180 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.975183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.975192 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:41:36.975223 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.975223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.975235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.975309 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:36.975496 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.976088 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.977473 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.978046 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.978134 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.978143 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.978312 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.978348 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:41:36.978415 18424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.975228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:40065 (local address 127.7.137.3:42607)
0619 02:41:36.975298 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:36.975300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:36.975441 (+ 141us) server_negotiation.cc:407] Connection header received
0619 02:41:36.975487 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:36.975488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:36.975499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:36.975550 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:41:36.975720 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.976229 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.977472 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.978072 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.978134 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.978144 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:36.978312 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:41:36.978346 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:41:36.978439 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.975011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:37667 (local address 127.7.137.1:59221)
0619 02:41:36.975073 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.975087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.975133 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.975314 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.975315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.975317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:36.975393 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.975395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.976168 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.976169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.977380 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.977383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.978142 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.978142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.978228 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.978235 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.978296 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.978341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":6}
I20260619 02:41:36.978446 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:36.975137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:42607 (local address 127.7.137.1:40065)
0619 02:41:36.975370 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:36.975378 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:36.975424 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:36.975574 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:41:36.975575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:41:36.975577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:41:36.975644 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.975645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.976309 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.976310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.977399 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:41:36.977401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:41:36.978142 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:41:36.978142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:41:36.978228 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:36.978235 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:41:36.978296 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:41:36.978341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":10,"thread_start_us":152,"threads_started":1}
I20260619 02:41:36.978804 18365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5731f86b3445db8771651cdbae9213" candidate_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e989542c614c65877ce11e8edc84cb" is_pre_election: true
I20260619 02:41:36.978881 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5731f86b3445db8771651cdbae9213" candidate_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b3cd033089403db6efc2075caf06b9" is_pre_election: true
I20260619 02:41:36.978956 18365 raft_consensus.cc:2454] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df38c27625b47d0b0b2ea6bdbf1caa7 in term 0.
I20260619 02:41:36.979053 18231 raft_consensus.cc:2454] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df38c27625b47d0b0b2ea6bdbf1caa7 in term 0.
I20260619 02:41:36.979176 18033 leader_election.cc:302] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df38c27625b47d0b0b2ea6bdbf1caa7, 65e989542c614c65877ce11e8edc84cb; no voters:
I20260619 02:41:36.979292 18432 raft_consensus.cc:2781] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:41:36.979348 18432 raft_consensus.cc:482] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:41:36.979373 18432 raft_consensus.cc:3037] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:36.980042 18432 raft_consensus.cc:504] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.980170 18432 leader_election.cc:288] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [CANDIDATE]: Term 1 election: Requested vote from peers a0b3cd033089403db6efc2075caf06b9 (127.7.137.2:37667), 65e989542c614c65877ce11e8edc84cb (127.7.137.3:42607)
I20260619 02:41:36.980310 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5731f86b3445db8771651cdbae9213" candidate_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b3cd033089403db6efc2075caf06b9"
I20260619 02:41:36.980351 18365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5731f86b3445db8771651cdbae9213" candidate_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e989542c614c65877ce11e8edc84cb"
I20260619 02:41:36.980415 18231 raft_consensus.cc:3037] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:36.980412 18365 raft_consensus.cc:3037] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:41:36.981201 18365 raft_consensus.cc:2454] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df38c27625b47d0b0b2ea6bdbf1caa7 in term 1.
I20260619 02:41:36.981212 18231 raft_consensus.cc:2454] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df38c27625b47d0b0b2ea6bdbf1caa7 in term 1.
I20260619 02:41:36.981369 18033 leader_election.cc:302] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df38c27625b47d0b0b2ea6bdbf1caa7, 65e989542c614c65877ce11e8edc84cb; no voters:
I20260619 02:41:36.981475 18432 raft_consensus.cc:2781] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:41:36.981657 18432 raft_consensus.cc:686] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4df38c27625b47d0b0b2ea6bdbf1caa7, State: Running, Role: LEADER
I20260619 02:41:36.981769 18432 consensus_queue.cc:237] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } }
I20260619 02:41:36.982578 17952 catalog_manager.cc:5495] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df38c27625b47d0b0b2ea6bdbf1caa7 (127.7.137.1). New cstate: current_term: 1 leader_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df38c27625b47d0b0b2ea6bdbf1caa7" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 } health_report { overall_health: UNKNOWN } } }
W20260619 02:41:37.012262 18144 tablet.cc:2369] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:41:37.021749 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:41:37.019513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.7.137.1:38307)
0619 02:41:37.019588 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:41:37.019589 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:41:37.019615 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:41:37.019648 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:41:37.019649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:41:37.019667 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:41:37.019720 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:41:37.019860 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:37.020321 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:37.021103 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:41:37.021284 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:41:37.021338 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:41:37.021460 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:41:37.021568 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:41:37.021627 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:41:37.021658 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 02:41:37.021675 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:41:37.023525 18365 raft_consensus.cc:1261] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 1 FOLLOWER]: Refusing update from remote peer 4df38c27625b47d0b0b2ea6bdbf1caa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:37.023626 18231 raft_consensus.cc:1261] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 1 FOLLOWER]: Refusing update from remote peer 4df38c27625b47d0b0b2ea6bdbf1caa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:41:37.023929 18432 consensus_queue.cc:1035] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0b3cd033089403db6efc2075caf06b9" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:37.024096 18435 consensus_queue.cc:1035] T 8a5731f86b3445db8771651cdbae9213 P 4df38c27625b47d0b0b2ea6bdbf1caa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e989542c614c65877ce11e8edc84cb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:41:37.026505 18440 mvcc.cc:205] Tried to move back new op lower bound from 7298403930205851648 to 7298403930037862400. Current Snapshot: MvccSnapshot[applied={T|T < 7298403930205851648}]
I20260619 02:41:37.027052 18439 mvcc.cc:205] Tried to move back new op lower bound from 7298403930205851648 to 7298403930037862400. Current Snapshot: MvccSnapshot[applied={T|T < 7298403930205851648}]
I20260619 02:41:37.028375 17951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:37.028463 17951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:41:37.029258 17951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc4a82b6673049bd9809fc0a8937f678: Sending DeleteTablet for 3 replicas of tablet 8a5731f86b3445db8771651cdbae9213
I20260619 02:41:37.029526 18345 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5731f86b3445db8771651cdbae9213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:35724
I20260619 02:41:37.029579 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5731f86b3445db8771651cdbae9213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:50468
I20260619 02:41:37.029767 18449 tablet_replica.cc:330] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: stopping tablet replica
I20260619 02:41:37.029757 18077 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5731f86b3445db8771651cdbae9213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:57758
I20260619 02:41:37.029793 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 18014
I20260619 02:41:37.029863 18449 raft_consensus.cc:2229] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:37.029950 18449 raft_consensus.cc:2258] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:37.030325 18210 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5731f86b3445db8771651cdbae9213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:41:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:50468
I20260619 02:41:37.030706 18449 ts_tablet_manager.cc:1905] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:37.030859 18448 tablet_replica.cc:330] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: stopping tablet replica
I20260619 02:41:37.031018 18448 raft_consensus.cc:2229] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:41:37.031168 18448 raft_consensus.cc:2258] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:41:37.031607 18448 ts_tablet_manager.cc:1905] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:41:37.032400 18449 ts_tablet_manager.cc:1918] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:37.032485 18449 log.cc:1196] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/8a5731f86b3445db8771651cdbae9213
I20260619 02:41:37.032809 18449 ts_tablet_manager.cc:1939] T 8a5731f86b3445db8771651cdbae9213 P a0b3cd033089403db6efc2075caf06b9: Deleting consensus metadata
I20260619 02:41:37.033241 17939 catalog_manager.cc:4841] TS a0b3cd033089403db6efc2075caf06b9 (127.7.137.2:37667): tablet 8a5731f86b3445db8771651cdbae9213 (table test-table [id=7c7ba2e310d24485a86b239948716255]) successfully deleted
W20260619 02:41:37.033403 17939 catalog_manager.cc:4808] TS a0b3cd033089403db6efc2075caf06b9 (127.7.137.2:37667): delete failed for tablet 8a5731f86b3445db8771651cdbae9213 because the tablet was not found. No further retry: Not found: Tablet not found: 8a5731f86b3445db8771651cdbae9213
I20260619 02:41:37.033448 18448 ts_tablet_manager.cc:1918] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:41:37.033517 18448 log.cc:1196] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/8a5731f86b3445db8771651cdbae9213
I20260619 02:41:37.033746 18448 ts_tablet_manager.cc:1939] T 8a5731f86b3445db8771651cdbae9213 P 65e989542c614c65877ce11e8edc84cb: Deleting consensus metadata
I20260619 02:41:37.034116 17939 catalog_manager.cc:4841] TS 65e989542c614c65877ce11e8edc84cb (127.7.137.3:42607): tablet 8a5731f86b3445db8771651cdbae9213 (table test-table [id=7c7ba2e310d24485a86b239948716255]) successfully deleted
W20260619 02:41:37.036162 17936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:38307 (error 108)
W20260619 02:41:37.036803 18423 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:41:37.036513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38307 (local address 127.0.0.1:57774)
0619 02:41:37.036569 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:41:37.036579 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:41:37.036617 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:41:37.036702 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:41:37.036913 17936 catalog_manager.cc:4584] TS 4df38c27625b47d0b0b2ea6bdbf1caa7 (127.7.137.1:38307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a5731f86b3445db8771651cdbae9213: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:41:37.037006 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 18148
I20260619 02:41:37.042024 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 18282
I20260619 02:41:37.046275 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 17920
2026-06-19T02:41:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1984 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:41:37 dist-test-slave-pdnb krb5kdc[18453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:41:37 dist-test-slave-pdnb krb5kdc[18453](info): set up 2 sockets
Jun 19 02:41:37 dist-test-slave-pdnb krb5kdc[18453](info): commencing operation
krb5kdc: starting...
W20260619 02:41:38.073745 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:41:38 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:41:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangeradmin_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangeradmin_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangerlookup_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangerlookup_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
I20260619 02:41:38.227257 7716 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_8LRmf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/postgres -l logfile start
2026-06-19 02:41:40.570 UTC [18483] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:41:40.571 UTC [18483] LOG: listening on IPv4 address "127.7.137.20", port 39823
2026-06-19 02:41:40.574 UTC [18483] LOG: listening on Unix socket "/tmp/.s.PGSQL.39823"
2026-06-19 02:41:40.595 UTC [18486] LOG: database system was shut down at 2026-06-19 02:41:39 UTC
2026-06-19 02:41:40.600 UTC [18483] LOG: database system is ready to accept connections
I20260619 02:41:41.557591 7716 mini_postgres.cc:96] Postgres bound to 39823
2026-06-19 02:41:41.563 UTC [18494] FATAL: database "slave" does not exist
127.7.137.20:39823 - accepting connections
I20260619 02:41:41.564152 7716 mini_ranger.cc:161] Starting Ranger...
I20260619 02:41:41.578373 7716 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:41:42.004702 7716 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:41:42.004830 7716 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin
2026-06-19 02:41:42,288 [I] DB FLAVOR :POSTGRES
2026-06-19 02:41:42,288 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:41:42,288 [I] Checking connection..
2026-06-19 02:41:42,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:41:42,545 [I] Checking connection passed.
2026-06-19 02:41:42,545 [I] --------- Verifying version history table ---------
2026-06-19 02:41:42,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:42,788 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:41:42,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:43,041 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:41:43,041 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:41:43,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:41:43.270 UTC [18558] WARNING: there is no transaction in progress
2026-06-19 02:41:43,276 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:41:43,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:41:43,518 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:41:43,518 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:41:43,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:41:43,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:41:44,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pdnb.c.gcp-upstream.internal','N') ;"
2026-06-19 02:41:44,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:41:44,485 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:41:44,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:41:44,744 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:41:44,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:41:44,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:41:45,211 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:41:45,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:41:46.148 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.163 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.178 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.192 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.207 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.227 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.287 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.294 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.309 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.319 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.328 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.341 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.350 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.356 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.362 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46.369 UTC [18666] WARNING: there is no transaction in progress
2026-06-19 02:41:46,964 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:41:46,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:41:47,217 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:41:47,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:41:47,473 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:41:47,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:41:47,733 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:41:47,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:41:47,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:41:48,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pdnb.c.gcp-upstream.internal';"
2026-06-19 02:41:48,451 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:41:48,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:41:48,705 [I] DB_PATCHES have already been applied
I20260619 02:41:48.711310 7716 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_8LRmf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:41:48.711374 7716 mini_ranger.cc:193] Using host: 127.7.137.20
I20260619 02:41:48.714475 7716 mini_ranger.cc:227] Ranger admin URL: http://127.7.137.20:36947
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:41:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:41:49.829049 7716 mini_ranger.cc:160] Time spent starting Ranger: real 8.265s user 0.003s sys 0.006s
Jun 19, 2026 2:42:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:42:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:42:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:42:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:42:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:42:10.544814 7716 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangerkms_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/rangerkms_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.137.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.137.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:42:10 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:42:10.622207 7716 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:42:10.631654 7716 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:42:11.437965 7716 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:42:11.438052 7716 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms
I20260619 02:42:11.438076 7716 mini_ranger_kms.cc:227] Using postgres at 127.7.137.20:39823
2026-06-19 02:42:11,615 [I] DB FLAVOR :POSTGRES
2026-06-19 02:42:11,615 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:42:11,615 [I] Checking connection
2026-06-19 02:42:11,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:42:11,880 [I] connection success
2026-06-19 02:42:11,881 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:42:11,881 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:42:11,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:42:12,134 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:42:12,135 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:42:12,135 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:42:12,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:39823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:42:12,378 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:42:12.844022 7716 mini_ranger_kms.cc:310] Created kms service
I20260619 02:42:13.064829 7716 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:42:13.108403 7716 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:42:13 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:13.476990 7716 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:42:13.477101 7716 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms:/tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:42:13.477134 7716 mini_ranger_kms.cc:240] Using host: 127.7.137.20
I20260619 02:42:13.482015 7716 mini_ranger_kms.cc:276] Ranger KMS PID: 18846
I20260619 02:42:13.482137 7716 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.137.20:40561
Jun 19, 2026 2:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:42:15.712852 7716 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.091s user 0.000s sys 0.160s
I20260619 02:42:15.712970 7716 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:42:15.713065 7716 mini_ranger_kms.cc:401] 127.7.137.20:40561/kms/v1/keys
Jun 19 02:42:15 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:42:16 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
Jun 19 02:42:16 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
Jun 19 02:42:18 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:42:18.951683 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:32863
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:32863
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:40561/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:19.032414 18897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:19.032652 18897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:19.032689 18897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:19.033851 18897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:42:19.033917 18897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:19.033952 18897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:19.033984 18897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:42:19.034006 18897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:42:19.035508 18897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.137.20:40561/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:32863
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:42:19.035766 18897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:19.035983 18897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:42:19.041874 18897 server_base.cc:1034] running on GCE node
W20260619 02:42:19.041977 18903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:19.042091 18902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:19.042125 18905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:19.042490 18897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:19.042788 18897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:19.043957 18897 hybrid_clock.cc:648] HybridClock initialized: now 1781836939043931 us; error 41 us; skew 500 ppm
Jun 19 02:42:19 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836939, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:19.053346 18897 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:42:19.053820 18897 webserver.cc:460] Webserver started at http://127.7.137.62:41189/ using document root <none> and password file <none>
I20260619 02:42:19.054049 18897 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:19.054107 18897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:19.054263 18897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:19 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836939, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:19.111714 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "345baad73ffc4726a905a5c67ecb8c81"
format_stamp: "Formatted at 2026-06-19 02:42:19 on dist-test-slave-pdnb"
server_key: "cc7e5e7f4dddcfead20083445bfc3b58"
server_key_iv: "ad38a798e6fadebdee4dc32ca2a0ab34"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:19.112165 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "345baad73ffc4726a905a5c67ecb8c81"
format_stamp: "Formatted at 2026-06-19 02:42:19 on dist-test-slave-pdnb"
server_key: "cc7e5e7f4dddcfead20083445bfc3b58"
server_key_iv: "ad38a798e6fadebdee4dc32ca2a0ab34"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:19.113945 18897 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:42:19.128159 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:19.128405 18897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:19.128518 18897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "345baad73ffc4726a905a5c67ecb8c81"
format_stamp: "Formatted at 2026-06-19 02:42:19 on dist-test-slave-pdnb"
server_key: "cc7e5e7f4dddcfead20083445bfc3b58"
server_key_iv: "ad38a798e6fadebdee4dc32ca2a0ab34"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:19.128626 18897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:19.157553 18897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:19.159768 18897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:19.159943 18897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:19.165416 18897 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:32863
I20260619 02:42:19.165448 18964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:32863 every 8 connection(s)
I20260619 02:42:19.165896 18897 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:42:19.166625 18965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:19.167701 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 18897
I20260619 02:42:19.167811 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:42:19.167995 7716 external_mini_cluster.cc:1428] Setting key e654745567f7e5c0f82aa96e71d61172
I20260619 02:42:19.169361 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: Bootstrap starting.
I20260619 02:42:19.170097 18965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:19.170441 18965 log.cc:824] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:19.171170 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: No bootstrap required, opened a new log
I20260619 02:42:19.173406 18965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } }
I20260619 02:42:19.173573 18965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:19.173635 18965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345baad73ffc4726a905a5c67ecb8c81, State: Initialized, Role: FOLLOWER
I20260619 02:42:19.173797 18965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } }
I20260619 02:42:19.173902 18965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:42:19.173955 18965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:42:19.174018 18965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:19.174719 18965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } }
I20260619 02:42:19.174830 18965 leader_election.cc:302] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345baad73ffc4726a905a5c67ecb8c81; no voters:
I20260619 02:42:19.175041 18965 leader_election.cc:288] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:42:19.175104 18970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:19.175340 18970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [term 1 LEADER]: Becoming Leader. State: Replica: 345baad73ffc4726a905a5c67ecb8c81, State: Running, Role: LEADER
I20260619 02:42:19.175447 18965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:42:19.175477 18970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } }
I20260619 02:42:19.175900 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345baad73ffc4726a905a5c67ecb8c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } } }
I20260619 02:42:19.176066 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [sys.catalog]: This master's current role is: LEADER
I20260619 02:42:19.176240 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345baad73ffc4726a905a5c67ecb8c81. Latest consensus state: current_term: 1 leader_uuid: "345baad73ffc4726a905a5c67ecb8c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345baad73ffc4726a905a5c67ecb8c81" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 32863 } } }
I20260619 02:42:19.176342 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81 [sys.catalog]: This master's current role is: LEADER
Jun 19 02:42:19 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:42:19.176493 18979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:42:19.179207 18979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:42:19.179674 18965 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:42:19.181453 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:19.168526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.7.137.62:32863)
0619 02:42:19.168832 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:19.168842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:19.168876 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:42:19.169420 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:19.169430 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:19.169499 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:19.169656 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:19.170476 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:19.171921 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:19.173069 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:19.173702 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:19.173862 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:19.177029 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:19.177033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:19.177033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:19.177044 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:19.179019 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:19.179413 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:19.179416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:19.179416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:19.179508 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:19.180178 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:19.180179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:19.180179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:19.180484 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:19.180981 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:19.181121 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:42:19.181241 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":75,"threads_started":1}
I20260619 02:42:19.184269 18979 catalog_manager.cc:1347] Generated new cluster ID: ee771b1075244e75870bd6345df43d40
I20260619 02:42:19.184417 18979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:42:19.211908 18979 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:42:19.212812 18979 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:42:19.215757 18979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: Generated new TSK 0
I20260619 02:42:19.215989 18979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:42:20.026046 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:40561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:20.144773 19010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:20.145121 19010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:20.145197 19010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:20.146494 19010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:20.146557 19010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:20.146631 19010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:42:20.148128 19010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.137.20:40561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:42:20.148377 19010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:20.148607 19010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:20.153993 19018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:20.154402 19016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:20.154824 19015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:20.155321 19010 server_base.cc:1034] running on GCE node
I20260619 02:42:20.155609 19010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:20.155853 19010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:20.157007 19010 hybrid_clock.cc:648] HybridClock initialized: now 1781836940156987 us; error 35 us; skew 500 ppm
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:20.160234 19010 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:42:20.160759 19010 webserver.cc:460] Webserver started at http://127.7.137.1:43961/ using document root <none> and password file <none>
I20260619 02:42:20.160988 19010 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:20.161058 19010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:20.161206 19010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:20.176331 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "428a7a50747d196310a3ef6bc8aa8c8b"
server_key_iv: "1996d7a820cb70988745feeb6de9ac96"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.176817 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "428a7a50747d196310a3ef6bc8aa8c8b"
server_key_iv: "1996d7a820cb70988745feeb6de9ac96"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.178328 19010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:42:20.195881 19027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.196121 19010 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260619 02:42:20.196252 19010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "428a7a50747d196310a3ef6bc8aa8c8b"
server_key_iv: "1996d7a820cb70988745feeb6de9ac96"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.197036 19010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:20.219945 19010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:20.222419 19010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:20.222637 19010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:20.222859 19010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:20.223258 19010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:20.223302 19010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.223332 19010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:20.223349 19010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.230806 19010 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:38965
I20260619 02:42:20.231324 19010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:42:20.234366 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 19010
I20260619 02:42:20.234479 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:42:20.234688 7716 external_mini_cluster.cc:1428] Setting key 68a0507a5e5733493a89c541e280a6a1
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:42:20.233160 19140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:38965 every 8 connection(s)
I20260619 02:42:20.245816 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.234171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:52691 (local address 127.7.137.62:32863)
0619 02:42:20.234586 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.234589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.234600 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:20.235410 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.235411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.235425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.235477 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:20.236230 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.236981 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.238423 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.238713 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.238771 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.241420 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:20.241422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:20.241422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:20.241432 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:20.243767 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.244251 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.244252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.244253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.244316 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.244578 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.244579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.244579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.244710 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:20.244783 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.245627 (+ 844us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.245711 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":52,"threads_started":1}
I20260619 02:42:20.247649 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.233871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:32863 (local address 127.7.137.1:52691)
0619 02:42:20.234317 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.234362 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.235228 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.235614 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.235618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.235856 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:20.236124 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.236130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.237086 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.237087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.238329 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.238331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.238841 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.238841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.238977 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.239584 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:20.239594 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:20.241313 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:20.243859 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.243866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.243869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.244168 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.244380 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.244380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.244381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.244502 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.244809 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:20.244815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:20.244992 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.245103 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.245259 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":98,"threads_started":1}
I20260619 02:42:20.248452 19141 heartbeater.cc:347] Connected to a master server at 127.7.137.62:32863
I20260619 02:42:20.248566 19141 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:20.248795 19141 heartbeater.cc:511] Master 127.7.137.62:32863 requested a full tablet report, sending...
I20260619 02:42:20.249368 18929 ts_manager.cc:194] Registered new tserver with Master: 1104ac0de34d4ea0b2cd07d4c6eb5bb0 (127.7.137.1:38965)
I20260619 02:42:20.250275 18929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:52691
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:42:20.280612 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:40561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:42:20.407964 19147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:20.408339 19147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:20.408422 19147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:20.410161 19147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:20.410269 19147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:20.410351 19147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:42:20.412552 19147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.137.20:40561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:42:20.412807 19147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:20.413077 19147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:20.419400 19154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:20.419453 19155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:20.419667 19157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:20.420251 19147 server_base.cc:1034] running on GCE node
I20260619 02:42:20.420547 19147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:20.420825 19147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:20.421998 19147 hybrid_clock.cc:648] HybridClock initialized: now 1781836940421972 us; error 42 us; skew 500 ppm
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:20.424576 19147 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:42:20.425132 19147 webserver.cc:460] Webserver started at http://127.7.137.2:45309/ using document root <none> and password file <none>
I20260619 02:42:20.425374 19147 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:20.425439 19147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:20.425582 19147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:20.442567 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "c82d905570cd4ffeaa5f4499a496d0b6"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "f4a86a78e25f128284f8c272df50fb8b"
server_key_iv: "5a80d4e5b1bd5d7c70345b9d04b21440"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.443037 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "c82d905570cd4ffeaa5f4499a496d0b6"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "f4a86a78e25f128284f8c272df50fb8b"
server_key_iv: "5a80d4e5b1bd5d7c70345b9d04b21440"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.444883 19147 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 02:42:20.459842 19165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.460108 19147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:42:20.460266 19147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "c82d905570cd4ffeaa5f4499a496d0b6"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "f4a86a78e25f128284f8c272df50fb8b"
server_key_iv: "5a80d4e5b1bd5d7c70345b9d04b21440"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.460464 19147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:20.498977 19147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:20.501543 19147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:20.501761 19147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:20.502058 19147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:20.502604 19147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:20.502696 19147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.502769 19147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:20.502833 19147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.509572 19147 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:44909
I20260619 02:42:20.510102 19147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:42:20.510450 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 19147
I20260619 02:42:20.510553 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:42:20.510732 7716 external_mini_cluster.cc:1428] Setting key de824052c87538a8aed2e858f57ad1a1
I20260619 02:42:20.513787 19278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:44909 every 8 connection(s)
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:42:20.528841 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.514945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:36673 (local address 127.7.137.62:32863)
0619 02:42:20.514991 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.514993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.517725 (+ 2732us) server_negotiation.cc:407] Connection header received
0619 02:42:20.518313 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.518314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.518327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.518393 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:20.519474 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.520285 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.522290 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.522606 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.522653 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.525470 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:20.525471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:20.525471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:20.525479 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:20.527242 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.527728 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.527729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.527729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.527786 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.528051 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.528052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.528053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.528182 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:20.528232 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.528613 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.528703 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:42:20.529604 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.514653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:32863 (local address 127.7.137.2:36673)
0619 02:42:20.517586 (+ 2933us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.517625 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.518199 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.518829 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.518834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.519097 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:20.519373 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.519379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.520886 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.520888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.522185 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.522188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.522749 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.522750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.522914 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.523653 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:20.523665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:20.525347 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:20.527336 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.527342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.527344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.527638 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.527852 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.527852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.527853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.527977 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.528264 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:20.528269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:20.528453 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.529188 (+ 735us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.529354 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2892,"thread_start_us":82,"threads_started":1}
I20260619 02:42:20.530337 19279 heartbeater.cc:347] Connected to a master server at 127.7.137.62:32863
I20260619 02:42:20.530443 19279 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:20.530627 19279 heartbeater.cc:511] Master 127.7.137.62:32863 requested a full tablet report, sending...
I20260619 02:42:20.531019 18929 ts_manager.cc:194] Registered new tserver with Master: c82d905570cd4ffeaa5f4499a496d0b6 (127.7.137.2:44909)
I20260619 02:42:20.531508 18929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:36673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:42:20.563638 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:40561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:42:20.690081 19284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:42:20.690902 19284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:42:20.691009 19284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:42:20.693694 19284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:42:20.693832 19284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:42:20.693939 19284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:42:20.696853 19284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.137.20:40561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:42:20.697343 19284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:42:20.697679 19284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:42:20.704054 19292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:42:20.704449 19290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:20.705672 19284 server_base.cc:1034] running on GCE node
W20260619 02:42:20.714432 19289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:42:20.714847 19284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:42:20.715173 19284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:42:20.722549 19284 hybrid_clock.cc:648] HybridClock initialized: now 1781836940722514 us; error 49 us; skew 500 ppm
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:42:20.726611 19284 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:42:20.727243 19284 webserver.cc:460] Webserver started at http://127.7.137.3:39487/ using document root <none> and password file <none>
I20260619 02:42:20.727555 19284 fs_manager.cc:359] Metadata directory not provided
I20260619 02:42:20.727674 19284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:42:20.727860 19284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:20.746361 19284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "038b57fd8b254c4c9c6e5a63689b8cbb"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "c3d47bacea9b69b00c011d2596a258fc"
server_key_iv: "cf0e64dce2f9eaf96863e3c4a383c741"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.746953 19284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "038b57fd8b254c4c9c6e5a63689b8cbb"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "c3d47bacea9b69b00c011d2596a258fc"
server_key_iv: "cf0e64dce2f9eaf96863e3c4a383c741"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.750901 19284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 02:42:20.768762 19299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.769032 19284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260619 02:42:20.769131 19284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "038b57fd8b254c4c9c6e5a63689b8cbb"
format_stamp: "Formatted at 2026-06-19 02:42:20 on dist-test-slave-pdnb"
server_key: "c3d47bacea9b69b00c011d2596a258fc"
server_key_iv: "cf0e64dce2f9eaf96863e3c4a383c741"
server_key_version: "kuduclusterkey@0"
I20260619 02:42:20.769232 19284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:42:20.789482 19284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:42:20.792048 19284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:42:20.792271 19284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:42:20.792551 19284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:42:20.793008 19284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:42:20.793102 19284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.793179 19284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:42:20.793238 19284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:42:20.799793 19284 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:33811
I20260619 02:42:20.800364 19284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:42:20.800575 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 19284
I20260619 02:42:20.800679 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:42:20.800905 7716 external_mini_cluster.cc:1428] Setting key e9fe5186c0b1439a262b370fbc8872d6
I20260619 02:42:20.803175 19412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:33811 every 8 connection(s)
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:42:20.815513 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.804698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:49543 (local address 127.7.137.62:32863)
0619 02:42:20.804771 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.804773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.804994 (+ 221us) server_negotiation.cc:407] Connection header received
0619 02:42:20.805690 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.805692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.805704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.805735 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:20.806604 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.807361 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.808888 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.809183 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.809232 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.812033 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:20.812034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:20.812035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:20.812042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:20.813816 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.814342 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.814344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.814344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.814403 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.814704 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.814705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.814706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.814844 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:20.814890 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.815328 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.815411 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:42:20.816306 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.804427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:32863 (local address 127.7.137.3:49543)
0619 02:42:20.804850 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.804900 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.805560 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.805922 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.805927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.806200 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:42:20.806489 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.806496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.807529 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.807530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.808783 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.808786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.809330 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.809331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.809477 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.810168 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:42:20.810178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:42:20.811896 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:42:20.813916 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.813923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.813926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.814246 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.814481 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:42:20.814482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:42:20.814483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:42:20.814619 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:42:20.814978 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:42:20.814984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:42:20.815162 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.815888 (+ 726us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.816070 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":83,"threads_started":1}
I20260619 02:42:20.817025 19413 heartbeater.cc:347] Connected to a master server at 127.7.137.62:32863
I20260619 02:42:20.817117 19413 heartbeater.cc:464] Registering TS with master...
I20260619 02:42:20.817299 19413 heartbeater.cc:511] Master 127.7.137.62:32863 requested a full tablet report, sending...
I20260619 02:42:20.817699 18929 ts_manager.cc:194] Registered new tserver with Master: 038b57fd8b254c4c9c6e5a63689b8cbb (127.7.137.3:33811)
I20260619 02:42:20.818181 18929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:49543
I20260619 02:42:20.823221 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:42:20.834236 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.824900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.7.137.62:32863)
0619 02:42:20.824942 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.824944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.825257 (+ 313us) server_negotiation.cc:407] Connection header received
0619 02:42:20.825358 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.825359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.825369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.825398 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:42:20.826165 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.826905 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.827991 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.828277 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.828338 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.829949 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:42:20.829950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:42:20.829950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:42:20.829957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:42:20.831596 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.831975 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.831976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.831977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.832027 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:42:20.832207 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:42:20.832207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:42:20.832208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:42:20.832336 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:42:20.833836 (+ 1500us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.834061 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.834138 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:42:20.836391 18929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:42:20.838775 18929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:42:20.856202 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.842651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44909 (local address 127.0.0.1:59818)
0619 02:42:20.843954 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.843963 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.844007 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.844260 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.844263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.844265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:20.844341 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.844342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.848131 (+ 3789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.848132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.854520 (+ 6388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.854522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.855866 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.855867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.855961 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.855977 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.856048 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.856103 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"thread_start_us":60,"threads_started":1}
I20260619 02:42:20.859917 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.843036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.7.137.1:38965)
0619 02:42:20.843823 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.843828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.845560 (+ 1732us) server_negotiation.cc:407] Connection header received
0619 02:42:20.845600 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.845602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.845618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.845681 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:20.846237 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.847163 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.852802 (+ 5639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.855669 (+ 2867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.855730 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.855788 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.858327 (+ 2539us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.858383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":51,"threads_started":1}
I20260619 02:42:20.860277 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.843167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.7.137.2:44909)
0619 02:42:20.843910 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.843913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.844054 (+ 141us) server_negotiation.cc:407] Connection header received
0619 02:42:20.844095 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.844097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.844113 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.844176 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:20.844416 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.848033 (+ 3617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.854644 (+ 6611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.855456 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.855531 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.855600 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.859875 (+ 4275us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.860194 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":81,"threads_started":1}
I20260619 02:42:20.856794 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.842805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38965 (local address 127.0.0.1:51988)
0619 02:42:20.845455 (+ 2650us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.845465 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.845506 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.846146 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.846147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.846148 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:20.846171 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.846172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.847258 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.847259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.852492 (+ 5233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.852495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.856546 (+ 4051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.856547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.856607 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.856611 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.856665 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.856708 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2634,"thread_start_us":51,"threads_started":1}
I20260619 02:42:20.858238 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.842183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:33811 (local address 127.0.0.1:36226)
0619 02:42:20.843600 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:20.843643 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:20.843702 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:20.846023 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:20.846024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:20.846026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:20.846085 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.846086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.850005 (+ 3919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.850007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.851433 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:20.851436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.858025 (+ 6589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.858025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:20.858099 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.858103 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:20.858134 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 02:42:20.858173 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1387,"thread_start_us":72,"threads_started":1}
I20260619 02:42:20.861688 19075 tablet_service.cc:1467] Processing CreateTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a (DEFAULT_TABLE table=test-table [id=68fda180dea34b0ea1251b4a8139a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:20.862020 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:20.842448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.7.137.3:33811)
0619 02:42:20.845815 (+ 3367us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:20.845821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:20.845830 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:42:20.845865 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:20.845866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:20.845880 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:20.845945 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:20.848791 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.849893 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.856888 (+ 6995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:20.857653 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:20.861708 (+ 4055us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:20.861810 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:20.861839 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 02:42:20.861941 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3327,"thread_start_us":65,"threads_started":1}
I20260619 02:42:20.862079 19075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b429cc0ed146b6b586dbb3582a7e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:20.862078 19213 tablet_service.cc:1467] Processing CreateTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a (DEFAULT_TABLE table=test-table [id=68fda180dea34b0ea1251b4a8139a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:20.862434 19213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b429cc0ed146b6b586dbb3582a7e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:20.862756 19347 tablet_service.cc:1467] Processing CreateTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a (DEFAULT_TABLE table=test-table [id=68fda180dea34b0ea1251b4a8139a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:42:20.863061 19347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b429cc0ed146b6b586dbb3582a7e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:42:20.867857 19432 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Bootstrap starting.
I20260619 02:42:20.868655 19432 tablet_bootstrap.cc:654] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:20.869040 19432 log.cc:824] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:20.869455 19433 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Bootstrap starting.
I20260619 02:42:20.870443 19433 tablet_bootstrap.cc:654] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:20.870813 19433 log.cc:824] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:20.871217 19434 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Bootstrap starting.
I20260619 02:42:20.872553 19434 tablet_bootstrap.cc:654] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Neither blocks nor log segments found. Creating new log.
I20260619 02:42:20.872923 19434 log.cc:824] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:42:20.873780 19432 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: No bootstrap required, opened a new log
I20260619 02:42:20.873850 19432 ts_tablet_manager.cc:1397] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 02:42:20.876360 19432 raft_consensus.cc:348] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.876524 19432 raft_consensus.cc:374] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:20.876554 19432 raft_consensus.cc:729] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1104ac0de34d4ea0b2cd07d4c6eb5bb0, State: Initialized, Role: FOLLOWER
I20260619 02:42:20.877146 19432 consensus_queue.cc:260] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.877310 19433 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: No bootstrap required, opened a new log
I20260619 02:42:20.877386 19433 ts_tablet_manager.cc:1397] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260619 02:42:20.877612 19432 ts_tablet_manager.cc:1428] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:42:20.877794 19141 heartbeater.cc:503] Master 127.7.137.62:32863 was elected leader, sending a full tablet report...
I20260619 02:42:20.879436 19433 raft_consensus.cc:348] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.879591 19433 raft_consensus.cc:374] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:20.879631 19433 raft_consensus.cc:729] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c82d905570cd4ffeaa5f4499a496d0b6, State: Initialized, Role: FOLLOWER
I20260619 02:42:20.879762 19433 consensus_queue.cc:260] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.880059 19433 ts_tablet_manager.cc:1428] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:42:20.880179 19279 heartbeater.cc:503] Master 127.7.137.62:32863 was elected leader, sending a full tablet report...
I20260619 02:42:20.883693 19434 tablet_bootstrap.cc:492] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: No bootstrap required, opened a new log
I20260619 02:42:20.883776 19434 ts_tablet_manager.cc:1397] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20260619 02:42:20.885298 19434 raft_consensus.cc:348] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.885458 19434 raft_consensus.cc:374] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:42:20.885500 19434 raft_consensus.cc:729] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 038b57fd8b254c4c9c6e5a63689b8cbb, State: Initialized, Role: FOLLOWER
I20260619 02:42:20.885624 19434 consensus_queue.cc:260] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:20.885950 19434 ts_tablet_manager.cc:1428] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:42:20.885983 19413 heartbeater.cc:503] Master 127.7.137.62:32863 was elected leader, sending a full tablet report...
Jun 19 02:42:20 dist-test-slave-pdnb krb5kdc[18453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836940, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
W20260619 02:42:20.983420 19142 tablet.cc:2369] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:21.000499 19438 raft_consensus.cc:482] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:42:21.000684 19438 raft_consensus.cc:504] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:21.001159 19438 leader_election.cc:288] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c82d905570cd4ffeaa5f4499a496d0b6 (127.7.137.2:44909), 038b57fd8b254c4c9c6e5a63689b8cbb (127.7.137.3:33811)
I20260619 02:42:21.004384 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.001566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:58395 (local address 127.7.137.3:33811)
0619 02:42:21.001623 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:21.001625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:21.001636 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:42:21.001741 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:21.001742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:21.001755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:21.001803 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:21.002043 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.002530 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.003632 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.004014 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.004104 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:21.004114 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:21.004271 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 02:42:21.004315 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:42:21.004395 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.001374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:33811 (local address 127.7.137.1:58395)
0619 02:42:21.001565 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.001579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:21.001642 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:21.001877 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:21.001878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:21.001879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:21.001960 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:21.001962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.002657 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.002658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:21.003529 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:21.003532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.004106 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.004107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:21.004180 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:21.004184 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:42:21.004255 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:42:21.004298 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
I20260619 02:42:21.004698 19364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b429cc0ed146b6b586dbb3582a7e5a" candidate_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" is_pre_election: true
I20260619 02:42:21.004885 19364 raft_consensus.cc:2454] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1104ac0de34d4ea0b2cd07d4c6eb5bb0 in term 0.
I20260619 02:42:21.005158 19029 leader_election.cc:302] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 038b57fd8b254c4c9c6e5a63689b8cbb, 1104ac0de34d4ea0b2cd07d4c6eb5bb0; no voters:
I20260619 02:42:21.005285 19438 raft_consensus.cc:2781] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:42:21.005326 19438 raft_consensus.cc:482] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:42:21.005344 19438 raft_consensus.cc:3037] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:21.006188 19438 raft_consensus.cc:504] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:21.006332 19438 leader_election.cc:288] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [CANDIDATE]: Term 1 election: Requested vote from peers c82d905570cd4ffeaa5f4499a496d0b6 (127.7.137.2:44909), 038b57fd8b254c4c9c6e5a63689b8cbb (127.7.137.3:33811)
I20260619 02:42:21.006534 19364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b429cc0ed146b6b586dbb3582a7e5a" candidate_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb"
I20260619 02:42:21.006609 19364 raft_consensus.cc:3037] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:21.007423 19364 raft_consensus.cc:2454] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1104ac0de34d4ea0b2cd07d4c6eb5bb0 in term 1.
I20260619 02:42:21.007598 19029 leader_election.cc:302] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 038b57fd8b254c4c9c6e5a63689b8cbb, 1104ac0de34d4ea0b2cd07d4c6eb5bb0; no voters:
I20260619 02:42:21.007678 19438 raft_consensus.cc:2781] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:42:21.007813 19438 raft_consensus.cc:686] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 1 LEADER]: Becoming Leader. State: Replica: 1104ac0de34d4ea0b2cd07d4c6eb5bb0, State: Running, Role: LEADER
I20260619 02:42:21.007917 19438 consensus_queue.cc:237] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } }
I20260619 02:42:21.008271 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.001541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44909 (local address 127.7.137.1:53245)
0619 02:42:21.004563 (+ 3022us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.004575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:42:21.004641 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:42:21.004841 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:42:21.004842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:42:21.004843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:42:21.004873 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:21.004874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.006489 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.006490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:21.007235 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:42:21.007237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.008037 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.008038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:42:21.008098 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:21.008101 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:42:21.008157 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:42:21.008194 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3008,"mutex_wait_us":19,"thread_start_us":76,"threads_started":1}
I20260619 02:42:21.008276 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.001744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53245 (local address 127.7.137.2:44909)
0619 02:42:21.001792 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:21.001794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:21.004614 (+ 2820us) server_negotiation.cc:407] Connection header received
0619 02:42:21.004707 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:21.004708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:21.004739 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:21.004768 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:42:21.004927 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.005591 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.007333 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.007926 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.008004 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:21.008014 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:21.008173 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:42:21.008213 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:42:21.008821 19233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b429cc0ed146b6b586dbb3582a7e5a" candidate_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" is_pre_election: true
I20260619 02:42:21.008962 19233 raft_consensus.cc:2454] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1104ac0de34d4ea0b2cd07d4c6eb5bb0 in term 0.
I20260619 02:42:21.008889 18929 catalog_manager.cc:5495] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1104ac0de34d4ea0b2cd07d4c6eb5bb0 (127.7.137.1). New cstate: current_term: 1 leader_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 } health_report { overall_health: UNKNOWN } } }
I20260619 02:42:21.009065 19232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b429cc0ed146b6b586dbb3582a7e5a" candidate_uuid: "1104ac0de34d4ea0b2cd07d4c6eb5bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c82d905570cd4ffeaa5f4499a496d0b6"
I20260619 02:42:21.009114 19232 raft_consensus.cc:3037] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:42:21.009999 19232 raft_consensus.cc:2454] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1104ac0de34d4ea0b2cd07d4c6eb5bb0 in term 1.
W20260619 02:42:21.016571 19281 tablet.cc:2369] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:42:21.028210 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.025676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.7.137.1:38965)
0619 02:42:21.025744 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:42:21.025746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:42:21.025756 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:42:21.025889 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:42:21.025891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:42:21.025920 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:42:21.026003 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:42:21.026148 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.026642 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.027410 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:42:21.027617 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:42:21.027696 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:42:21.027875 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:42:21.027990 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:42:21.028085 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:42:21.028105 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 02:42:21.028126 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:42:21.030100 19364 raft_consensus.cc:1261] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 1 FOLLOWER]: Refusing update from remote peer 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:21.030100 19232 raft_consensus.cc:1261] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 1 FOLLOWER]: Refusing update from remote peer 1104ac0de34d4ea0b2cd07d4c6eb5bb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:42:21.030431 19438 consensus_queue.cc:1035] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c82d905570cd4ffeaa5f4499a496d0b6" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:21.030567 19443 consensus_queue.cc:1035] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "038b57fd8b254c4c9c6e5a63689b8cbb" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:42:21.034387 19448 mvcc.cc:205] Tried to move back new op lower bound from 7298404110456479744 to 7298404110368976896. Current Snapshot: MvccSnapshot[applied={T|T < 7298404110456479744}]
I20260619 02:42:21.036917 18928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:21.037009 18928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:42:21.037892 18928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 345baad73ffc4726a905a5c67ecb8c81: Sending DeleteTablet for 3 replicas of tablet f7b429cc0ed146b6b586dbb3582a7e5a
I20260619 02:42:21.038309 19213 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:21 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:59818
I20260619 02:42:21.038517 19075 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:21 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:51988
I20260619 02:42:21.038951 19347 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b429cc0ed146b6b586dbb3582a7e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:42:21 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:36226
I20260619 02:42:21.040884 19455 tablet_replica.cc:330] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: stopping tablet replica
I20260619 02:42:21.041049 19455 raft_consensus.cc:2229] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:21.041219 19455 raft_consensus.cc:2258] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:21.041736 19455 ts_tablet_manager.cc:1905] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:21.043952 19455 ts_tablet_manager.cc:1918] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:42:21.044090 19455 log.cc:1196] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/f7b429cc0ed146b6b586dbb3582a7e5a
I20260619 02:42:21.044438 19455 ts_tablet_manager.cc:1939] T f7b429cc0ed146b6b586dbb3582a7e5a P c82d905570cd4ffeaa5f4499a496d0b6: Deleting consensus metadata
I20260619 02:42:21.044983 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 19010
I20260619 02:42:21.045068 18916 catalog_manager.cc:4841] TS c82d905570cd4ffeaa5f4499a496d0b6 (127.7.137.2:44909): tablet f7b429cc0ed146b6b586dbb3582a7e5a (table test-table [id=68fda180dea34b0ea1251b4a8139a051]) successfully deleted
I20260619 02:42:21.045084 19456 tablet_replica.cc:330] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0: stopping tablet replica
I20260619 02:42:21.045197 19456 raft_consensus.cc:2229] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:42:21.045346 19456 raft_consensus.cc:2258] T f7b429cc0ed146b6b586dbb3582a7e5a P 1104ac0de34d4ea0b2cd07d4c6eb5bb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:21.045647 19457 tablet_replica.cc:330] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: stopping tablet replica
I20260619 02:42:21.045750 19457 raft_consensus.cc:2229] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:42:21.045858 19457 raft_consensus.cc:2258] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:42:21.049297 19457 ts_tablet_manager.cc:1905] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:42:21.051011 19457 ts_tablet_manager.cc:1918] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 02:42:21.051013 18916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:38965 (error 108)
I20260619 02:42:21.051156 19457 log.cc:1196] T f7b429cc0ed146b6b586dbb3582a7e5a P 038b57fd8b254c4c9c6e5a63689b8cbb: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/f7b429cc0ed146b6b586dbb3582a7e5a
I20260619 02:42:21.051669 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 19147
I20260619 02:42:21.052306 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:42:21.051801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38965 (local address 127.0.0.1:52010)
0619 02:42:21.052229 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 02:42:21.052275 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
W20260619 02:42:21.053391 18916 catalog_manager.cc:4584] TS 1104ac0de34d4ea0b2cd07d4c6eb5bb0 (127.7.137.1:38965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7b429cc0ed146b6b586dbb3582a7e5a: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38965: connect: Connection refused (error 111)
I20260619 02:42:21.056656 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 19284
I20260619 02:42:21.062320 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 18897
I20260619 02:42:21.067799 7716 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:42:21.402158 7716 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:42:21.402266 7716 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:42:21.746801 7716 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:42:21.747 UTC [18483] LOG: received smart shutdown request
2026-06-19 02:42:21.749 UTC [18483] LOG: background worker "logical replication launcher" (PID 18492) exited with exit code 1
2026-06-19 02:42:21.750 UTC [18487] LOG: shutting down
2026-06-19 02:42:21.894 UTC [18483] LOG: database system is shut down
2026-06-19T02:42:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44953 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:42:22 dist-test-slave-pdnb krb5kdc[19467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:42:22 dist-test-slave-pdnb krb5kdc[19467](info): set up 2 sockets
Jun 19 02:42:22 dist-test-slave-pdnb krb5kdc[19467](info): commencing operation
krb5kdc: starting...
W20260619 02:42:23.048636 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:42:23 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:42:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangeradmin_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangeradmin_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangerlookup_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangerlookup_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
I20260619 02:42:23.199951 7716 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_8LRmf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/postgres -l logfile start
2026-06-19 02:42:25.514 UTC [19497] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:42:25.514 UTC [19497] LOG: listening on IPv4 address "127.7.137.20", port 38475
2026-06-19 02:42:25.517 UTC [19497] LOG: listening on Unix socket "/tmp/.s.PGSQL.38475"
2026-06-19 02:42:25.551 UTC [19500] LOG: database system was shut down at 2026-06-19 02:42:24 UTC
2026-06-19 02:42:25.555 UTC [19497] LOG: database system is ready to accept connections
I20260619 02:42:26.500582 7716 mini_postgres.cc:96] Postgres bound to 38475
2026-06-19 02:42:26.506 UTC [19508] FATAL: database "slave" does not exist
127.7.137.20:38475 - accepting connections
I20260619 02:42:26.507408 7716 mini_ranger.cc:161] Starting Ranger...
I20260619 02:42:26.522317 7716 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:42:26.972311 7716 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:42:26.972396 7716 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin
2026-06-19 02:42:27,224 [I] DB FLAVOR :POSTGRES
2026-06-19 02:42:27,224 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:42:27,224 [I] Checking connection..
2026-06-19 02:42:27,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:42:27,481 [I] Checking connection passed.
2026-06-19 02:42:27,481 [I] --------- Verifying version history table ---------
2026-06-19 02:42:27,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:42:27,716 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:42:27,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:42:27,977 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:42:27,977 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:42:27,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:42:28.208 UTC [19572] WARNING: there is no transaction in progress
2026-06-19 02:42:28,214 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:42:28,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:42:28,469 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:42:28,470 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:42:28,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:42:28,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:42:28,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pdnb.c.gcp-upstream.internal','N') ;"
2026-06-19 02:42:29,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:42:29,427 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:42:29,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:42:29,652 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:42:29,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:42:29,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:42:30,166 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:42:30,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:42:31.095 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.111 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.126 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.139 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.154 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.170 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.222 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.228 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.241 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.250 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.260 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.273 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.282 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.287 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.293 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31.299 UTC [19680] WARNING: there is no transaction in progress
2026-06-19 02:42:31,882 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:42:31,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:42:32,115 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:42:32,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:42:32,379 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:42:32,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:42:32,605 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:42:32,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:42:32,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:42:33,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pdnb.c.gcp-upstream.internal';"
2026-06-19 02:42:33,303 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:42:33,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:42:33,560 [I] DB_PATCHES have already been applied
I20260619 02:42:33.565505 7716 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_8LRmf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:42:33.565568 7716 mini_ranger.cc:193] Using host: 127.7.137.20
I20260619 02:42:33.568557 7716 mini_ranger.cc:227] Ranger admin URL: http://127.7.137.20:48453
Jun 19, 2026 2:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:42:34.674371 7716 mini_ranger.cc:160] Time spent starting Ranger: real 8.167s user 0.000s sys 0.008s
Jun 19, 2026 2:42:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:42:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 2:42:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:42:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:42:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:42:55.236742 7716 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.137.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.137.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangerkms_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.137.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/rangerkms_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.137.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.137.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.137.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/HTTP_127.7.137.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:42:55 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836975, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:42:55.315600 7716 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:42:55.324626 7716 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:42:56.099582 7716 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:42:56.099653 7716 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms
I20260619 02:42:56.099666 7716 mini_ranger_kms.cc:227] Using postgres at 127.7.137.20:38475
2026-06-19 02:42:56,260 [I] DB FLAVOR :POSTGRES
2026-06-19 02:42:56,260 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:42:56,260 [I] Checking connection
2026-06-19 02:42:56,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:42:56,488 [I] connection success
2026-06-19 02:42:56,489 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:42:56,489 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:42:56,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:42:56,716 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:42:56,716 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:42:56,716 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:42:56,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.137.20:38475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:42:56,965 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:42:57.400139 7716 mini_ranger_kms.cc:310] Created kms service
I20260619 02:42:57.591320 7716 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:42:57.635766 7716 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:42:57 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836975, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:42:58.018488 7716 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:42:58.018747 7716 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms:/tmp/dist-test-task_8LRmf/build/release/bin/postgresql.jar:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_8LRmf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:42:58.018772 7716 mini_ranger_kms.cc:240] Using host: 127.7.137.20
I20260619 02:42:58.021960 7716 mini_ranger_kms.cc:276] Ranger KMS PID: 19860
I20260619 02:42:58.022035 7716 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.137.20:50137
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:42:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:43:00.260711 7716 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.945s user 0.010s sys 0.133s
I20260619 02:43:00.260844 7716 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:43:00.260900 7716 mini_ranger_kms.cc:401] 127.7.137.20:50137/kms/v1/keys
Jun 19 02:43:00 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836980, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:43:00 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836980, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
Jun 19 02:43:01 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836980, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.137.20@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
Jun 19 02:43:03 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:43:03.371644 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45373
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45373
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:03.453022 19911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:03.453233 19911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:03.453254 19911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:03.454403 19911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:03.454448 19911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:03.454463 19911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:03.454501 19911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:03.454527 19911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:03.456151 19911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.137.20:50137/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45373
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:03.456398 19911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:03.456617 19911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:03.461690 19917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:03.461638 19919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:03.461650 19916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:03.461699 19911 server_base.cc:1034] running on GCE node
I20260619 02:43:03.462172 19911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:03.462450 19911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:03.463611 19911 hybrid_clock.cc:648] HybridClock initialized: now 1781836983463606 us; error 51 us; skew 500 ppm
Jun 19 02:43:03 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:03.466060 19911 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:03.466472 19911 webserver.cc:460] Webserver started at http://127.7.137.62:43903/ using document root <none> and password file <none>
I20260619 02:43:03.466698 19911 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:03.466765 19911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:03.466917 19911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:43:03 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:03.510354 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "49646e4daeb243be9c2d53b4e4c617ae"
format_stamp: "Formatted at 2026-06-19 02:43:03 on dist-test-slave-pdnb"
server_key: "28f9f3bfd94afc812d4e8b18ac52dd84"
server_key_iv: "6168955f859a604d81a88a1f9accec7d"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:03.510838 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "49646e4daeb243be9c2d53b4e4c617ae"
format_stamp: "Formatted at 2026-06-19 02:43:03 on dist-test-slave-pdnb"
server_key: "28f9f3bfd94afc812d4e8b18ac52dd84"
server_key_iv: "6168955f859a604d81a88a1f9accec7d"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:03.512686 19911 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 02:43:03.527697 19926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:03.527937 19911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:03.528043 19911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "49646e4daeb243be9c2d53b4e4c617ae"
format_stamp: "Formatted at 2026-06-19 02:43:03 on dist-test-slave-pdnb"
server_key: "28f9f3bfd94afc812d4e8b18ac52dd84"
server_key_iv: "6168955f859a604d81a88a1f9accec7d"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:03.528134 19911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:03.554776 19911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:03.556689 19911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:03.556885 19911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:03.561223 19911 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45373
I20260619 02:43:03.561271 19978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45373 every 8 connection(s)
I20260619 02:43:03.561621 19911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:03.562266 19979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:03.564632 19979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Bootstrap starting.
I20260619 02:43:03.565274 19979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:03.565569 19979 log.cc:824] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:03.566286 19979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: No bootstrap required, opened a new log
I20260619 02:43:03.567410 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 19911
I20260619 02:43:03.567544 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:43:03.567737 7716 external_mini_cluster.cc:1428] Setting key 02d3d995f360d6ab0764a1328678f7ae
I20260619 02:43:03.568619 19979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:03.568836 19979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:03.568902 19979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49646e4daeb243be9c2d53b4e4c617ae, State: Initialized, Role: FOLLOWER
I20260619 02:43:03.569070 19979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:03.569172 19979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:03.569219 19979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:03.569269 19979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:03.569814 19979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:03.569931 19979 leader_election.cc:302] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49646e4daeb243be9c2d53b4e4c617ae; no voters:
I20260619 02:43:03.570119 19979 leader_election.cc:288] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:03.570170 19984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:03.570281 19984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 LEADER]: Becoming Leader. State: Replica: 49646e4daeb243be9c2d53b4e4c617ae, State: Running, Role: LEADER
I20260619 02:43:03.570405 19979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:03.570427 19984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:03.571022 19984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49646e4daeb243be9c2d53b4e4c617ae. Latest consensus state: current_term: 1 leader_uuid: "49646e4daeb243be9c2d53b4e4c617ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } } }
I20260619 02:43:03.571029 19986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49646e4daeb243be9c2d53b4e4c617ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } } }
I20260619 02:43:03.571228 19984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:03.571241 19986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:03.571938 19979 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:43:03.571966 19993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:03.574971 19993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:03.577509 19993 catalog_manager.cc:1347] Generated new cluster ID: 16078a05b0d34b8ab4f94766db27dcd9
I20260619 02:43:03.577631 19993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:43:03 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:03.584120 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:03.568323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.7.137.62:45373)
0619 02:43:03.568513 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:03.568523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:03.568561 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:43:03.568670 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:03.568675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:03.568761 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:03.568906 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:03.569645 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:03.571000 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:03.572121 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:03.575272 (+ 3151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:03.576894 (+ 1622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:03.579498 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:03.579503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:03.579503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:03.579515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:03.582126 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:03.582515 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:03.582518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:03.582518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:03.582625 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:03.582804 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:03.582804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:03.582805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:03.583108 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:03.583227 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:03.583758 (+ 531us) server_negotiation.cc:299] Negotiation successful
0619 02:43:03.583897 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260619 02:43:03.591100 19993 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:03.591703 19993 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:03.602662 19993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Generated new TSK 0
I20260619 02:43:03.602865 19993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:43:04.307317 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:04.393671 20024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:04.393874 20024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:04.393896 20024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:04.395107 20024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:04.395147 20024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:04.395174 20024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:04.396638 20024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:04.396905 20024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:04.397133 20024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:04.402151 20029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.402379 20032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.402417 20030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:04.402902 20024 server_base.cc:1034] running on GCE node
I20260619 02:43:04.403113 20024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:04.403395 20024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:04.404553 20024 hybrid_clock.cc:648] HybridClock initialized: now 1781836984404534 us; error 38 us; skew 500 ppm
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:04.407552 20024 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:04.408157 20024 webserver.cc:460] Webserver started at http://127.7.137.1:42587/ using document root <none> and password file <none>
I20260619 02:43:04.408466 20024 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:04.408545 20024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:04.408699 20024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:04.423108 20024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "7157a75664e8b5c409b011476e0814ee"
server_key_iv: "5617c1a9a80c47d1023755a07fb75351"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.423527 20024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "7157a75664e8b5c409b011476e0814ee"
server_key_iv: "5617c1a9a80c47d1023755a07fb75351"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.429137 20024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260619 02:43:04.443653 20039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.443898 20024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:04.443992 20024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "7157a75664e8b5c409b011476e0814ee"
server_key_iv: "5617c1a9a80c47d1023755a07fb75351"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.444084 20024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:04.463758 20024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:04.467211 20024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:04.467435 20024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:04.467691 20024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:04.468114 20024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:04.468209 20024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.468324 20024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:04.468400 20024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.478608 20024 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:33137
I20260619 02:43:04.479202 20024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:04.480084 20154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:33137 every 8 connection(s)
I20260619 02:43:04.484579 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20024
I20260619 02:43:04.484910 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:04.485145 7716 external_mini_cluster.cc:1428] Setting key 5b7d8d7c4ec29fee239a3b6d44223ec4
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:04.495946 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:04.482554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.1:38957)
0619 02:43:04.483739 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:04.483793 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:04.484512 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:04.485320 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:04.485325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:04.485609 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:04.485925 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:04.485934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.487088 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.487089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:04.488532 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:04.488536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.489220 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.489221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:04.489384 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:04.490099 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:04.490110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:04.492007 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:04.494237 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:04.494246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:04.494248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:04.494555 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:04.494779 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:04.494780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:04.494780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:04.494915 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:04.495208 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:04.495227 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:04.495417 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 02:43:04.495532 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:43:04.495696 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"thread_start_us":87,"threads_started":1}
I20260619 02:43:04.496292 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:04.482878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:38957 (local address 127.7.137.62:45373)
0619 02:43:04.484908 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:04.484911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:04.484920 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:04.484949 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:04.484950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:04.485017 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:04.485109 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:04.486052 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.486917 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.488649 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.489027 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.489123 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:04.492163 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:04.492164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:04.492165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:04.492174 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:04.494127 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:04.494643 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:04.494644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:04.494644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:04.494712 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:04.494984 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:04.494985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:04.494985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:04.495122 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:04.495173 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:04.496094 (+ 921us) server_negotiation.cc:299] Negotiation successful
0619 02:43:04.496184 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1997,"thread_start_us":82,"threads_started":1}
I20260619 02:43:04.497015 20155 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:04.497159 20155 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:04.497388 20155 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:04.498023 19943 ts_manager.cc:194] Registered new tserver with Master: e3178d9fdc0b4cf4a73fef7a45d6e930 (127.7.137.1:33137)
I20260619 02:43:04.499066 19943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:38957
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:43:04.531869 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:04.650635 20161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:04.651032 20161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:04.651119 20161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:04.653095 20161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:04.653419 20161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:04.653510 20161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:04.655757 20161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:04.656239 20161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:04.656545 20161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:04.663022 20169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.663417 20168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.663626 20171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:04.665959 20161 server_base.cc:1034] running on GCE node
I20260619 02:43:04.666181 20161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:04.666452 20161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:04.669741 20161 hybrid_clock.cc:648] HybridClock initialized: now 1781836984669719 us; error 38 us; skew 500 ppm
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:04.673177 20161 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:04.673717 20161 webserver.cc:460] Webserver started at http://127.7.137.2:38503/ using document root <none> and password file <none>
I20260619 02:43:04.673978 20161 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:04.674027 20161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:04.674154 20161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:04.687613 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "1e2ec6e7e2a2467b8a189324e9de38a5"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "afa79e95d2f88a6c40f977436510ba16"
server_key_iv: "3469d7d559d0889a0593fd2ff1eae407"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.688143 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "1e2ec6e7e2a2467b8a189324e9de38a5"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "afa79e95d2f88a6c40f977436510ba16"
server_key_iv: "3469d7d559d0889a0593fd2ff1eae407"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.693145 20161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.001s
I20260619 02:43:04.718006 20178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.718294 20161 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260619 02:43:04.718392 20161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "1e2ec6e7e2a2467b8a189324e9de38a5"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "afa79e95d2f88a6c40f977436510ba16"
server_key_iv: "3469d7d559d0889a0593fd2ff1eae407"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.718492 20161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:04.735116 20161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:04.737589 20161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:04.737783 20161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:04.738075 20161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:04.738476 20161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:04.738534 20161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.738574 20161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:04.738600 20161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.746493 20161 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:45703
I20260619 02:43:04.746948 20161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:04.747290 20291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:45703 every 8 connection(s)
I20260619 02:43:04.752466 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20161
I20260619 02:43:04.752597 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:04.752830 7716 external_mini_cluster.cc:1428] Setting key 858db4bff8d2a0466ad35d694f3a903c
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:43:04.766827 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:04.749699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:57477 (local address 127.7.137.62:45373)
0619 02:43:04.749754 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:04.749756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:04.749766 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:04.749792 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:04.749793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:04.749808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:04.749876 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:04.751481 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.752264 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.754254 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.754566 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.754611 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:04.761241 (+ 6630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:04.761243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:04.761243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:04.761253 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:04.763019 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:04.765385 (+ 2366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:04.765386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:04.765387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:04.765454 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:04.765955 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:04.765956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:04.765957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:04.766104 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:04.766158 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:04.766542 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 02:43:04.766655 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
I20260619 02:43:04.769486 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:04.748018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.2:57477)
0619 02:43:04.748461 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:04.748506 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:04.749228 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:04.750756 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:04.750760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:04.751029 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:04.751341 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:04.751347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.752837 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.752838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:04.754144 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:04.754147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:04.754725 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:04.754726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:04.754867 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:04.755472 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:04.755483 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:04.761104 (+ 5621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:04.764951 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:04.764960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:04.764963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:04.765284 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:04.765712 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:04.765713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:04.765714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:04.765869 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:04.766190 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:04.766196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:04.766388 (+ 192us) client_negotiation.cc:769] Sending connection context
0619 02:43:04.767185 (+ 797us) client_negotiation.cc:240] Negotiation successful
0619 02:43:04.769235 (+ 2050us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":79,"threads_started":1}
I20260619 02:43:04.770226 20292 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:04.770340 20292 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:04.770552 20292 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:04.770990 19943 ts_manager.cc:194] Registered new tserver with Master: 1e2ec6e7e2a2467b8a189324e9de38a5 (127.7.137.2:45703)
I20260619 02:43:04.771505 19943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:57477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:43:04.784801 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:04.894747 20297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:04.895022 20297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:04.895071 20297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:04.896832 20297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:04.896891 20297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:04.896945 20297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:04.899248 20297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:04.899500 20297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:04.899780 20297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:04.906482 20303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.906488 20304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:04.906579 20306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:04.907232 20297 server_base.cc:1034] running on GCE node
I20260619 02:43:04.907572 20297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:04.907869 20297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:04.909058 20297 hybrid_clock.cc:648] HybridClock initialized: now 1781836984909026 us; error 48 us; skew 500 ppm
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:04.912413 20297 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:04.912992 20297 webserver.cc:460] Webserver started at http://127.7.137.3:43457/ using document root <none> and password file <none>
I20260619 02:43:04.913282 20297 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:04.913343 20297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:04.913473 20297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:43:04 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:04.939921 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "23709b85614d4cf8856eca097bfcc8a7"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "eb69e02e1a33011ef89ab9c47023b94a"
server_key_iv: "2834190efc789bbb203de6be38ab310c"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.940439 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "23709b85614d4cf8856eca097bfcc8a7"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "eb69e02e1a33011ef89ab9c47023b94a"
server_key_iv: "2834190efc789bbb203de6be38ab310c"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.949627 20297 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260619 02:43:04.971873 20313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.972080 20297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:04.972193 20297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "23709b85614d4cf8856eca097bfcc8a7"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "eb69e02e1a33011ef89ab9c47023b94a"
server_key_iv: "2834190efc789bbb203de6be38ab310c"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:04.972304 20297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:04.988413 20297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:04.990579 20297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:04.990749 20297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:04.990998 20297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:04.991320 20297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:04.991376 20297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:04.991415 20297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:04.991443 20297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.004838 20297 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45823
I20260619 02:43:05.005313 20297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:05.006076 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45823 every 8 connection(s)
I20260619 02:43:05.010973 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20297
I20260619 02:43:05.011094 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:05.011276 7716 external_mini_cluster.cc:1428] Setting key c143ca0430192b34d2b093ee5a099360
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:05.017642 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.006872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:50625 (local address 127.7.137.62:45373)
0619 02:43:05.006920 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:05.006922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:05.007463 (+ 541us) server_negotiation.cc:407] Connection header received
0619 02:43:05.008186 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:05.008187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:05.008200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:05.008231 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:05.009091 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.009814 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.011297 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.011597 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.011650 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.014279 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:05.014280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:05.014280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:05.014288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:05.015967 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.016485 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.016486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.016487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.016546 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.016866 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.016867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.016867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.017013 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:05.017056 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:05.017458 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 02:43:05.017547 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:05.018198 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.007030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.3:50625)
0619 02:43:05.007338 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:05.007381 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:05.008014 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:05.008423 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:05.008427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:05.008679 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:05.008980 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.008986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.009939 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.009940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.011192 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.011195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.011754 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.011755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.011888 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.012568 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:05.012578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:05.014165 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:05.016065 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.016071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.016074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.016392 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.016617 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.016618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.016618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.016783 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.017104 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:05.017110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:05.017299 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 02:43:05.017785 (+ 486us) client_negotiation.cc:240] Negotiation successful
0619 02:43:05.017949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":103,"threads_started":1}
I20260619 02:43:05.018625 20427 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:05.018725 20427 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.018922 20427 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:05.019294 19943 ts_manager.cc:194] Registered new tserver with Master: 23709b85614d4cf8856eca097bfcc8a7 (127.7.137.3:45823)
I20260619 02:43:05.019788 19943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:50625
I20260619 02:43:05.022771 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:05.023435 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20024
I20260619 02:43:05.030855 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20161
I20260619 02:43:05.037076 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20297
I20260619 02:43:05.042783 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 19911
I20260619 02:43:05.049598 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45373
--webserver_interface=127.7.137.62
--webserver_port=43903
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45373
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:05.131320 20430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:05.131570 20430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:05.131609 20430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:05.132815 20430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:05.132874 20430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:05.132900 20430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:05.132930 20430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:05.132956 20430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:05.134495 20430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45373
--ranger_config_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=43903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:05.134783 20430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:05.135017 20430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:05.140787 20435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:05.140818 20436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.140856 20430 server_base.cc:1034] running on GCE node
W20260619 02:43:05.141028 20438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.141295 20430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:05.141611 20430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:05.142782 20430 hybrid_clock.cc:648] HybridClock initialized: now 1781836985142769 us; error 27 us; skew 500 ppm
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:05.145653 20430 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:05.146083 20430 webserver.cc:460] Webserver started at http://127.7.137.62:43903/ using document root <none> and password file <none>
I20260619 02:43:05.146317 20430 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:05.146373 20430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:05.190702 20430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:05.191488 20446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.191682 20430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:05.191792 20430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "49646e4daeb243be9c2d53b4e4c617ae"
format_stamp: "Formatted at 2026-06-19 02:43:03 on dist-test-slave-pdnb"
server_key: "28f9f3bfd94afc812d4e8b18ac52dd84"
server_key_iv: "6168955f859a604d81a88a1f9accec7d"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:05.192251 20430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:05.205058 20430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:05.207126 20430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:05.207274 20430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:05.212280 20430 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45373
I20260619 02:43:05.212308 20498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45373 every 8 connection(s)
I20260619 02:43:05.212798 20430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:05.214093 20499 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:43:05.214742 20499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Bootstrap starting.
I20260619 02:43:05.216068 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20430
I20260619 02:43:05.216396 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:33137
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:45373
--webserver_port=42587
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:05.223479 20499 log.cc:824] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:05.225283 20499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:05.225585 20499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Bootstrap complete.
I20260619 02:43:05.227933 20499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:05.228199 20499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49646e4daeb243be9c2d53b4e4c617ae, State: Initialized, Role: FOLLOWER
I20260619 02:43:05.228406 20499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:05.228492 20499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:05.228541 20499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:05.228580 20499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:43:05.229650 20499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:05.229772 20499 leader_election.cc:302] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49646e4daeb243be9c2d53b4e4c617ae; no voters:
I20260619 02:43:05.229987 20499 leader_election.cc:288] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:43:05.230022 20503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:43:05.230137 20503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [term 2 LEADER]: Becoming Leader. State: Replica: 49646e4daeb243be9c2d53b4e4c617ae, State: Running, Role: LEADER
I20260619 02:43:05.230228 20503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } }
I20260619 02:43:05.230257 20499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:05.230541 20505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "49646e4daeb243be9c2d53b4e4c617ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } } }
I20260619 02:43:05.230693 20505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:05.230934 20507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:05.231016 20504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49646e4daeb243be9c2d53b4e4c617ae. Latest consensus state: current_term: 2 leader_uuid: "49646e4daeb243be9c2d53b4e4c617ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49646e4daeb243be9c2d53b4e4c617ae" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45373 } } }
I20260619 02:43:05.231107 20504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:05.231489 20507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:05.231664 20507 catalog_manager.cc:1259] Loaded cluster ID: 16078a05b0d34b8ab4f94766db27dcd9
I20260619 02:43:05.231731 20507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:05.231913 20499 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:43:05.235858 20507 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:05.236366 20507 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Loaded TSK: 0
I20260619 02:43:05.236703 20507 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:43:05.328634 20501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:05.328919 20501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:05.328959 20501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:05.330183 20501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:05.330250 20501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:05.330296 20501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:05.332311 20501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:33137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=42587
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:05.332551 20501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:05.332839 20501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:05.341749 20539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.343063 20501 server_base.cc:1034] running on GCE node
W20260619 02:43:05.344869 20536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:05.345125 20537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.345433 20501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:05.345721 20501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:05.346902 20501 hybrid_clock.cc:648] HybridClock initialized: now 1781836985346875 us; error 44 us; skew 500 ppm
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:05.350224 20501 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:05.350764 20501 webserver.cc:460] Webserver started at http://127.7.137.1:42587/ using document root <none> and password file <none>
I20260619 02:43:05.351058 20501 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:05.351138 20501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:05.381882 20501 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260619 02:43:05.382953 20547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.383208 20501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:05.383306 20501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "7157a75664e8b5c409b011476e0814ee"
server_key_iv: "5617c1a9a80c47d1023755a07fb75351"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:05.383796 20501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:05.402077 20501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:05.404300 20501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:05.404450 20501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:05.404762 20501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:05.405145 20501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:05.405181 20501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.405205 20501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:05.405216 20501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.412313 20501 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:33137
I20260619 02:43:05.412786 20501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:05.412931 20660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:33137 every 8 connection(s)
I20260619 02:43:05.413993 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20501
I20260619 02:43:05.414335 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:45703
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:45373
--webserver_port=38503
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:05.428377 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.413935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.1:47609)
0619 02:43:05.414466 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:05.414509 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:05.415261 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:05.415948 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:05.415952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:05.416153 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:05.416339 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.416343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.419303 (+ 2960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.419304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.420587 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.420591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.421569 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.421570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.421733 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.422372 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:05.422382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:05.423878 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:05.426643 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.426650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.426652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.426938 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.427204 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.427205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.427205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.427316 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.427747 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:05.427752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:05.427894 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:43:05.427990 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:43:05.428141 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":89,"threads_started":1}
I20260619 02:43:05.429400 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.414102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:47609 (local address 127.7.137.62:45373)
0619 02:43:05.414372 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:05.414382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:05.415425 (+ 1043us) server_negotiation.cc:407] Connection header received
0619 02:43:05.415582 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:05.415590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:05.415653 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:05.415787 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:05.416457 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.419164 (+ 2707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.420789 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.421435 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.423102 (+ 1667us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.424000 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:05.424004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:05.424005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:05.424019 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:05.426515 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.427029 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.427033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.427033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.427134 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.427379 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.427380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.427380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.427667 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:05.428999 (+ 1332us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:05.429075 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:43:05.429198 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":122,"threads_started":1}
I20260619 02:43:05.430008 20661 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:05.430135 20661 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.430316 20661 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.431119 20661 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45373 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:43:05.431182 20661 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.431239 20661 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:05.431607 20661 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.431679 20661 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.431993 20661 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:43:05.518448 20665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:05.518774 20665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:05.518828 20665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:05.520642 20665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:05.520757 20665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:05.520819 20665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:05.522981 20665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:45703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=38503
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:05.523363 20665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:05.523629 20665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:05.530821 20671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:05.530830 20673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.531242 20665 server_base.cc:1034] running on GCE node
W20260619 02:43:05.532891 20670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.533195 20665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:05.533556 20665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:05.534772 20665 hybrid_clock.cc:648] HybridClock initialized: now 1781836985534756 us; error 36 us; skew 500 ppm
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:05.537647 20665 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:05.538195 20665 webserver.cc:460] Webserver started at http://127.7.137.2:38503/ using document root <none> and password file <none>
I20260619 02:43:05.538512 20665 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:05.538606 20665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:05.561229 20665 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.001s
I20260619 02:43:05.562448 20681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.562695 20665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:05.562794 20665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "1e2ec6e7e2a2467b8a189324e9de38a5"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "afa79e95d2f88a6c40f977436510ba16"
server_key_iv: "3469d7d559d0889a0593fd2ff1eae407"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:05.563285 20665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:05.598140 20665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:05.600827 20665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:05.601006 20665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:05.601289 20665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:05.601707 20665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:05.601761 20665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.601794 20665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:05.601861 20665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.609354 20665 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:45703
I20260619 02:43:05.609931 20665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:05.611907 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20665
I20260619 02:43:05.612233 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.137.20:50137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:45823
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:45373
--webserver_port=43457
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:05.620900 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.611210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:40119 (local address 127.7.137.62:45373)
0619 02:43:05.611274 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:05.611276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:05.611499 (+ 223us) server_negotiation.cc:407] Connection header received
0619 02:43:05.612237 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:05.612239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:05.612255 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:05.612320 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:05.613157 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.613845 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.614857 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.615073 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.615160 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.617718 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:05.617719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:05.617720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:05.617728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:05.619315 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.619820 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.619821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.619821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.619883 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.620207 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.620208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.620209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.620337 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:05.620370 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:05.620671 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:43:05.620806 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:05.620946 20794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:45703 every 8 connection(s)
I20260619 02:43:05.621166 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.611217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.2:40119)
0619 02:43:05.611377 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:05.611413 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:05.612110 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:05.612459 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:05.612464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:05.612714 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:05.613044 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.613050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.613959 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.613965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.614760 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.614762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.615191 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.615193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.615308 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.615877 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:05.615887 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:05.617605 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:05.619400 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.619407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.619410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.619722 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.619954 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.619955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.619955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.620102 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.620408 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:05.620413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:05.620538 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 02:43:05.620653 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:43:05.620834 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260619 02:43:05.621765 20795 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:05.621882 20795 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.622071 20795 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.622704 20795 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45373 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:43:05.622786 20795 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.622859 20795 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:05.623142 20795 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.623215 20795 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.623473 20795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:43:05.738868 20798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:05.739219 20798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:05.739271 20798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:05.741221 20798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:05.741289 20798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:05.741334 20798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:05.743708 20798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.137.20:50137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:45823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=43457
--tserver_master_addrs=127.7.137.62:45373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:05.744117 20798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:05.744441 20798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:05.751013 20807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:05.751216 20805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:05.751040 20804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:05.751084 20798 server_base.cc:1034] running on GCE node
I20260619 02:43:05.751557 20798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:05.751850 20798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:05.753005 20798 hybrid_clock.cc:648] HybridClock initialized: now 1781836985752988 us; error 35 us; skew 500 ppm
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:05.755774 20798 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:05.756285 20798 webserver.cc:460] Webserver started at http://127.7.137.3:43457/ using document root <none> and password file <none>
I20260619 02:43:05.756557 20798 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:05.756657 20798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:05.775060 20798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:43:05.776964 20815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.777262 20798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:05.777367 20798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "23709b85614d4cf8856eca097bfcc8a7"
format_stamp: "Formatted at 2026-06-19 02:43:04 on dist-test-slave-pdnb"
server_key: "eb69e02e1a33011ef89ab9c47023b94a"
server_key_iv: "2834190efc789bbb203de6be38ab310c"
server_key_version: "kuduclusterkey@0"
I20260619 02:43:05.777969 20798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:05.791949 20798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:05.794052 20798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:05.794221 20798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:05.794490 20798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:05.794907 20798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:05.794965 20798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.795035 20798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:05.795063 20798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:05.802491 20798 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45823
I20260619 02:43:05.802902 20798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:05.803514 20928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45823 every 8 connection(s)
I20260619 02:43:05.809321 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 20798
Jun 19 02:43:05 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:05.814270 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.803963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:53465 (local address 127.7.137.62:45373)
0619 02:43:05.804022 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:05.804024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:05.804034 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:05.804637 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:05.804639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:05.804653 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:05.804775 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:05.805497 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.806254 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.807805 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.808104 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.808209 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.810742 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:05.810744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:05.810745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:05.810752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:05.812642 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.813168 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.813169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.813169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.813229 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.813508 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.813509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.813509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.813649 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:05.813692 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:05.814066 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 02:43:05.814144 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:43:05.815053 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.803687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45373 (local address 127.7.137.3:53465)
0619 02:43:05.803854 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:05.803898 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:05.804502 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:05.804854 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:05.804859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:05.805086 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:05.805370 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.805376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.806374 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.806375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.807693 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:05.807696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.808243 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.808245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:05.808395 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.809068 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:05.809079 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:05.810621 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:05.812759 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.812766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.812769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.813076 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.813301 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:05.813301 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:05.813302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:05.813435 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:05.813740 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:05.813746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:05.813915 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 02:43:05.814640 (+ 725us) client_negotiation.cc:240] Negotiation successful
0619 02:43:05.814817 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260619 02:43:05.815538 20929 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45373
I20260619 02:43:05.815675 20929 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.815903 20929 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.816905 20929 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45373 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:43:05.816992 20929 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.817070 20929 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:05.817412 20929 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:05.817492 20929 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
W20260619 02:43:05.817847 20929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:43:05.818362 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:05.809911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.7.137.62:45373)
0619 02:43:05.810308 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:05.810310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:05.810318 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:05.810340 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:05.810341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:05.810354 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:05.810423 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:05.811242 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.811998 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.813417 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:05.813722 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:05.813780 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:05.815246 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:05.815248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:05.815249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:05.815255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:05.816977 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.817385 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.817386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.817386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.817444 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:05.817670 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:05.817671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:05.817671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:05.817799 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:05.817957 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:05.818177 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 02:43:05.818265 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":52,"threads_started":1}
Jun 19 02:43:06 dist-test-slave-pdnb krb5kdc[19467](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:06.432152 20661 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:06.432336 20661 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:06.432773 20463 ts_manager.cc:194] Registered new tserver with Master: e3178d9fdc0b4cf4a73fef7a45d6e930 (127.7.137.1:33137)
I20260619 02:43:06.433799 20463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:47609
I20260619 02:43:06.623668 20795 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:06.623809 20795 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:06.624250 20463 ts_manager.cc:194] Registered new tserver with Master: 1e2ec6e7e2a2467b8a189324e9de38a5 (127.7.137.2:45703)
I20260619 02:43:06.624814 20463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:40119
Jun 19 02:43:06 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
I20260619 02:43:06.818058 20929 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:06.818341 20929 heartbeater.cc:511] Master 127.7.137.62:45373 requested a full tablet report, sending...
I20260619 02:43:06.818713 20463 ts_manager.cc:194] Registered new tserver with Master: 23709b85614d4cf8856eca097bfcc8a7 (127.7.137.3:45823)
I20260619 02:43:06.819303 20463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:53465
I20260619 02:43:06.819319 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:06.828639 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.820325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.7.137.62:45373)
0619 02:43:06.820511 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:06.820513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:06.820522 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:06.820545 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:06.820546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:06.820557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:06.820636 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:06.821380 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.822052 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.823111 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.823398 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.823477 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.824990 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:06.824991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:06.824991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:06.824999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:06.826990 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:06.827412 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:06.827413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:06.827413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:06.827478 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:06.827691 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:06.827692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:06.827693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:06.827827 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:06.827940 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:06.828414 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 02:43:06.828510 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260619 02:43:06.831244 20463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:06.833534 20463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:06.845613 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45703 (local address 127.0.0.1:34910)
0619 02:43:06.841157 (+ 4473us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:06.841167 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:06.841208 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:06.841518 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:06.841519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:06.841520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:06.841575 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.841577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.842849 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.842850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.844257 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.844260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.845304 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.845305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.845393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.845407 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:43:06.845474 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:43:06.845525 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4451,"mutex_wait_us":47,"thread_start_us":80,"threads_started":1}
I20260619 02:43:06.846185 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.7.137.2:45703)
0619 02:43:06.841317 (+ 4340us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:06.841322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:06.841331 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:06.841362 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:06.841364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:06.841378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:06.841445 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:06.841654 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.842750 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.844357 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.845142 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.845211 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.845270 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:06.846061 (+ 791us) server_negotiation.cc:299] Negotiation successful
0619 02:43:06.846106 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4304,"thread_start_us":54,"threads_started":1}
I20260619 02:43:06.846241 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33137 (local address 127.0.0.1:49470)
0619 02:43:06.837168 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:06.837208 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:06.837281 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:06.841079 (+ 3798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:06.841084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:06.841087 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:06.841184 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.841187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.842923 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.842924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.844516 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.844518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.846002 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.846003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.846063 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.846067 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:06.846115 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:43:06.846154 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":29,"thread_start_us":98,"threads_started":1}
I20260619 02:43:06.846521 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.7.137.1:33137)
0619 02:43:06.840885 (+ 3918us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:06.840891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:06.840902 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:06.840940 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:06.840942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:06.840958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:06.841045 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:06.841293 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.842815 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.844591 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.845844 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.845916 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.845965 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:06.846397 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 02:43:06.846444 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3876,"thread_start_us":70,"threads_started":1}
I20260619 02:43:06.847133 20729 tablet_service.cc:1467] Processing CreateTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 (DEFAULT_TABLE table=test-table [id=407787ce7cd9409cb0e45a2327b49991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:06.847393 20582 tablet_service.cc:1467] Processing CreateTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 (DEFAULT_TABLE table=test-table [id=407787ce7cd9409cb0e45a2327b49991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:06.847539 20729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1843cfac6e3460ba8c5324e864f7ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:06.847735 20582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1843cfac6e3460ba8c5324e864f7ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:06.854568 20962 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Bootstrap starting.
I20260619 02:43:06.855270 20962 tablet_bootstrap.cc:654] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:06.855487 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45823 (local address 127.0.0.1:47702)
0619 02:43:06.837378 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:06.837386 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:06.837417 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:06.846906 (+ 9489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:06.846907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:06.846909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:06.846947 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.846949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.853068 (+ 6119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.853069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.854236 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:06.854239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.855252 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.855254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:06.855324 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.855329 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:43:06.855361 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 02:43:06.855404 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260619 02:43:06.855800 20962 log.cc:824] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:06.855881 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:06.836759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47702 (local address 127.7.137.3:45823)
0619 02:43:06.846733 (+ 9974us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:06.846738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:06.846749 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:06.846784 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:06.846786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:06.846801 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:06.847016 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:06.847100 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.852976 (+ 5876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.854347 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:06.855132 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:06.855660 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:06.855731 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:06.855757 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 02:43:06.855802 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9935,"thread_start_us":60,"threads_started":1}
I20260619 02:43:06.856247 20963 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Bootstrap starting.
I20260619 02:43:06.856624 20849 tablet_service.cc:1467] Processing CreateTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 (DEFAULT_TABLE table=test-table [id=407787ce7cd9409cb0e45a2327b49991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:06.856884 20963 tablet_bootstrap.cc:654] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:06.856980 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1843cfac6e3460ba8c5324e864f7ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:06.857182 20963 log.cc:824] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:06.857914 20962 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: No bootstrap required, opened a new log
I20260619 02:43:06.857987 20962 ts_tablet_manager.cc:1397] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:43:06.859788 20963 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: No bootstrap required, opened a new log
I20260619 02:43:06.859859 20963 ts_tablet_manager.cc:1397] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 02:43:06.860555 20966 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Bootstrap starting.
I20260619 02:43:06.861068 20962 raft_consensus.cc:348] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.861312 20966 tablet_bootstrap.cc:654] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:06.861653 20966 log.cc:824] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:06.863749 20963 raft_consensus.cc:348] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.863960 20963 raft_consensus.cc:374] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:06.863941 20962 raft_consensus.cc:374] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:06.864005 20962 raft_consensus.cc:729] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2ec6e7e2a2467b8a189324e9de38a5, State: Initialized, Role: FOLLOWER
I20260619 02:43:06.864078 20963 raft_consensus.cc:729] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3178d9fdc0b4cf4a73fef7a45d6e930, State: Initialized, Role: FOLLOWER
I20260619 02:43:06.864132 20962 consensus_queue.cc:260] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.864322 20963 consensus_queue.cc:260] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.864429 20962 ts_tablet_manager.cc:1428] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260619 02:43:06.864573 20963 ts_tablet_manager.cc:1428] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:06.864593 20795 heartbeater.cc:503] Master 127.7.137.62:45373 was elected leader, sending a full tablet report...
I20260619 02:43:06.864776 20661 heartbeater.cc:503] Master 127.7.137.62:45373 was elected leader, sending a full tablet report...
I20260619 02:43:06.866839 20966 tablet_bootstrap.cc:492] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: No bootstrap required, opened a new log
I20260619 02:43:06.866915 20966 ts_tablet_manager.cc:1397] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 02:43:06.869431 20966 raft_consensus.cc:348] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.869594 20966 raft_consensus.cc:374] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:06.869626 20966 raft_consensus.cc:729] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23709b85614d4cf8856eca097bfcc8a7, State: Initialized, Role: FOLLOWER
I20260619 02:43:06.869791 20966 consensus_queue.cc:260] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:06.870056 20966 ts_tablet_manager.cc:1428] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:06.870087 20929 heartbeater.cc:503] Master 127.7.137.62:45373 was elected leader, sending a full tablet report...
W20260619 02:43:06.914322 20662 tablet.cc:2369] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:43:06 dist-test-slave-pdnb krb5kdc[19467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for HTTP/127.7.137.20@KRBTEST.COM
W20260619 02:43:07.054275 20930 tablet.cc:2369] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:07.111435 20796 tablet.cc:2369] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:07.119107 20969 raft_consensus.cc:482] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:07.119288 20969 raft_consensus.cc:504] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:07.119707 20969 leader_election.cc:288] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23709b85614d4cf8856eca097bfcc8a7 (127.7.137.3:45823), 1e2ec6e7e2a2467b8a189324e9de38a5 (127.7.137.2:45703)
I20260619 02:43:07.124403 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.120184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:49047 (local address 127.7.137.2:45703)
0619 02:43:07.120244 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:07.120247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:07.120765 (+ 518us) server_negotiation.cc:407] Connection header received
0619 02:43:07.120881 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:07.120882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:07.120895 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:07.120927 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:07.121161 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.121900 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.123417 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.123958 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.124011 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:07.124023 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:07.124283 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:43:07.124328 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:43:07.124953 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.119932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45703 (local address 127.7.137.1:49047)
0619 02:43:07.120663 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:07.120674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:07.120798 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:07.121001 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:07.121002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:07.121004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:07.121083 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:07.121085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.122014 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.122016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:07.123306 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:07.123310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.124105 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.124106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:07.124189 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:07.124193 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:07.124798 (+ 605us) client_negotiation.cc:240] Negotiation successful
0619 02:43:07.124847 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":108,"threads_started":1}
I20260619 02:43:07.125490 20735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1843cfac6e3460ba8c5324e864f7ee1" candidate_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" is_pre_election: true
I20260619 02:43:07.125663 20735 raft_consensus.cc:2454] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3178d9fdc0b4cf4a73fef7a45d6e930 in term 0.
I20260619 02:43:07.126027 20548 leader_election.cc:302] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2ec6e7e2a2467b8a189324e9de38a5, e3178d9fdc0b4cf4a73fef7a45d6e930; no voters:
I20260619 02:43:07.126164 20969 raft_consensus.cc:2781] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:07.126223 20969 raft_consensus.cc:482] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:07.126245 20969 raft_consensus.cc:3037] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:07.127202 20969 raft_consensus.cc:504] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:07.127337 20969 leader_election.cc:288] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [CANDIDATE]: Term 1 election: Requested vote from peers 23709b85614d4cf8856eca097bfcc8a7 (127.7.137.3:45823), 1e2ec6e7e2a2467b8a189324e9de38a5 (127.7.137.2:45703)
I20260619 02:43:07.127707 20735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1843cfac6e3460ba8c5324e864f7ee1" candidate_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5"
I20260619 02:43:07.127792 20735 raft_consensus.cc:3037] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:07.128700 20735 raft_consensus.cc:2454] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3178d9fdc0b4cf4a73fef7a45d6e930 in term 1.
I20260619 02:43:07.128952 20548 leader_election.cc:302] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2ec6e7e2a2467b8a189324e9de38a5, e3178d9fdc0b4cf4a73fef7a45d6e930; no voters:
I20260619 02:43:07.129073 20969 raft_consensus.cc:2781] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:07.129241 20969 raft_consensus.cc:686] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [term 1 LEADER]: Becoming Leader. State: Replica: e3178d9fdc0b4cf4a73fef7a45d6e930, State: Running, Role: LEADER
I20260619 02:43:07.129334 20969 consensus_queue.cc:237] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } }
I20260619 02:43:07.130151 20463 catalog_manager.cc:5495] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3178d9fdc0b4cf4a73fef7a45d6e930 (127.7.137.1). New cstate: current_term: 1 leader_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 } health_report { overall_health: UNKNOWN } } }
I20260619 02:43:07.133426 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.120573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:60063 (local address 127.7.137.3:45823)
0619 02:43:07.120614 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:07.120616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:07.129943 (+ 9327us) server_negotiation.cc:407] Connection header received
0619 02:43:07.130056 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:07.130057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:07.130070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:07.130101 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:07.130298 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.131033 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.132463 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.133031 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.133084 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:07.133097 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:07.133310 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:43:07.133354 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:07.133788 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.120515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45823 (local address 127.7.137.1:60063)
0619 02:43:07.129871 (+ 9356us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:07.129884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:07.129975 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:07.130164 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:07.130165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:07.130166 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:07.130214 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:07.130215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.131269 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.131269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:07.132363 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:07.132367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.133155 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.133156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:07.133226 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:07.133229 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:07.133674 (+ 445us) client_negotiation.cc:240] Negotiation successful
0619 02:43:07.133720 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9341,"thread_start_us":81,"threads_started":1}
I20260619 02:43:07.134298 20877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1843cfac6e3460ba8c5324e864f7ee1" candidate_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23709b85614d4cf8856eca097bfcc8a7" is_pre_election: true
I20260619 02:43:07.134457 20877 raft_consensus.cc:2454] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3178d9fdc0b4cf4a73fef7a45d6e930 in term 0.
I20260619 02:43:07.134318 20883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1843cfac6e3460ba8c5324e864f7ee1" candidate_uuid: "e3178d9fdc0b4cf4a73fef7a45d6e930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23709b85614d4cf8856eca097bfcc8a7"
I20260619 02:43:07.134730 20883 raft_consensus.cc:3037] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:07.135663 20883 raft_consensus.cc:2454] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3178d9fdc0b4cf4a73fef7a45d6e930 in term 1.
I20260619 02:43:07.194594 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.191215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.7.137.1:33137)
0619 02:43:07.191345 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:07.191347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:07.191353 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:43:07.191387 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:07.191388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:07.191408 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:07.191487 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:07.191631 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.192366 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.193507 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:07.193792 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:07.193852 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:07.194043 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:07.194186 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:07.194262 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:07.194491 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 02:43:07.194513 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260619 02:43:07.196593 20735 raft_consensus.cc:1261] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 1 FOLLOWER]: Refusing update from remote peer e3178d9fdc0b4cf4a73fef7a45d6e930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:07.196625 20883 raft_consensus.cc:1261] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 1 FOLLOWER]: Refusing update from remote peer e3178d9fdc0b4cf4a73fef7a45d6e930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:07.196965 20969 consensus_queue.cc:1035] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23709b85614d4cf8856eca097bfcc8a7" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:07.197121 20969 consensus_queue.cc:1035] T d1843cfac6e3460ba8c5324e864f7ee1 P e3178d9fdc0b4cf4a73fef7a45d6e930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2ec6e7e2a2467b8a189324e9de38a5" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:07.201149 20978 mvcc.cc:205] Tried to move back new op lower bound from 7298404299554410496 to 7298404299282305024. Current Snapshot: MvccSnapshot[applied={T|T < 7298404299554410496}]
I20260619 02:43:07.201548 20980 mvcc.cc:205] Tried to move back new op lower bound from 7298404299554410496 to 7298404299282305024. Current Snapshot: MvccSnapshot[applied={T|T < 7298404299554410496}]
I20260619 02:43:07.203547 20981 mvcc.cc:205] Tried to move back new op lower bound from 7298404299554410496 to 7298404299282305024. Current Snapshot: MvccSnapshot[applied={T|T < 7298404299554410496}]
I20260619 02:43:07.204804 20462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:07.204905 20462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:07.205709 20462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49646e4daeb243be9c2d53b4e4c617ae: Sending DeleteTablet for 3 replicas of tablet d1843cfac6e3460ba8c5324e864f7ee1
I20260619 02:43:07.206081 20729 tablet_service.cc:1514] Processing DeleteTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:07 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:34910
I20260619 02:43:07.206137 20582 tablet_service.cc:1514] Processing DeleteTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:07 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:49470
I20260619 02:43:07.206535 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet d1843cfac6e3460ba8c5324e864f7ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:07 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47702
I20260619 02:43:07.207146 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20501
I20260619 02:43:07.208992 20991 tablet_replica.cc:330] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: stopping tablet replica
I20260619 02:43:07.209110 20991 raft_consensus.cc:2229] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:07.209237 20991 raft_consensus.cc:2258] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:07.209735 20991 ts_tablet_manager.cc:1905] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:07.211766 20991 ts_tablet_manager.cc:1918] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:07.211843 20991 log.cc:1196] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/d1843cfac6e3460ba8c5324e864f7ee1
I20260619 02:43:07.212173 20991 ts_tablet_manager.cc:1939] T d1843cfac6e3460ba8c5324e864f7ee1 P 23709b85614d4cf8856eca097bfcc8a7: Deleting consensus metadata
I20260619 02:43:07.213001 20989 tablet_replica.cc:330] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: stopping tablet replica
W20260619 02:43:07.213109 20447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:33137 (error 108)
I20260619 02:43:07.213173 20989 raft_consensus.cc:2229] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:07.213285 20989 raft_consensus.cc:2258] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:07.213723 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20665
I20260619 02:43:07.213740 20989 ts_tablet_manager.cc:1905] T d1843cfac6e3460ba8c5324e864f7ee1 P 1e2ec6e7e2a2467b8a189324e9de38a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:07.216120 20450 catalog_manager.cc:4841] TS 23709b85614d4cf8856eca097bfcc8a7 (127.7.137.3:45823): tablet d1843cfac6e3460ba8c5324e864f7ee1 (table test-table [id=407787ce7cd9409cb0e45a2327b49991]) successfully deleted
I20260619 02:43:07.216339 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:07.216211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33137 (local address 127.0.0.1:49490)
0619 02:43:07.216263 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:07.216301 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:43:07.216454 20447 catalog_manager.cc:4584] TS e3178d9fdc0b4cf4a73fef7a45d6e930 (127.7.137.1:33137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1843cfac6e3460ba8c5324e864f7ee1: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33137: connect: Connection refused (error 111)
W20260619 02:43:07.219415 20957 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:43:07.218859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45703 (local address 127.0.0.1:34924)
0619 02:43:07.218900 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:07.218910 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:07.218945 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:07.219370 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.2:45703: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 02:43:07.219571 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20798
I20260619 02:43:07.224277 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 20430
I20260619 02:43:07.229535 7716 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:43:07.564842 7716 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:43:07.564945 7716 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:43:07.909726 7716 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:43:07.909 UTC [19497] LOG: received smart shutdown request
2026-06-19 02:43:07.913 UTC [19497] LOG: background worker "logical replication launcher" (PID 19506) exited with exit code 1
2026-06-19 02:43:07.913 UTC [19501] LOG: shutting down
2026-06-19 02:43:08.075 UTC [19497] LOG: database system is shut down
2026-06-19T02:43:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46185 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:08 dist-test-slave-pdnb krb5kdc[21006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:08 dist-test-slave-pdnb krb5kdc[21006](info): set up 2 sockets
Jun 19 02:43:08 dist-test-slave-pdnb krb5kdc[21006](info): commencing operation
krb5kdc: starting...
W20260619 02:43:09.236649 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:43:09.353098 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36497
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:09.433554 21019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:09.433784 21019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:09.433820 21019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:09.434958 21019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:09.435024 21019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:09.435048 21019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:09.435070 21019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:09.435101 21019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:09.436551 21019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:09.436846 21019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:09.437067 21019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:09.441891 21024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:09.442009 21019 server_base.cc:1034] running on GCE node
W20260619 02:43:09.441883 21025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:09.441883 21027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:09.442363 21019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:09.442600 21019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:09.443744 21019 hybrid_clock.cc:648] HybridClock initialized: now 1781836989443724 us; error 43 us; skew 500 ppm
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:09.446213 21019 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:09.446642 21019 webserver.cc:460] Webserver started at http://127.7.137.62:44305/ using document root <none> and password file <none>
I20260619 02:43:09.446866 21019 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:09.446911 21019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:09.447000 21019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:09.448074 21019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "d78e1ba5db4645a181a556d47e7cc26c"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "171f24cd73e5d5428cb86e00f004d435"
server_key_iv: "52179097f3eb502cfce652a408e7700d"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.448426 21019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "d78e1ba5db4645a181a556d47e7cc26c"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "171f24cd73e5d5428cb86e00f004d435"
server_key_iv: "52179097f3eb502cfce652a408e7700d"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.449777 21019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:09.450505 21034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:09.450732 21019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:09.450805 21019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "d78e1ba5db4645a181a556d47e7cc26c"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "171f24cd73e5d5428cb86e00f004d435"
server_key_iv: "52179097f3eb502cfce652a408e7700d"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.450893 21019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:09.480937 21019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:09.483042 21019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:09.483199 21019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:09.487434 21019 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36497
I20260619 02:43:09.487509 21086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36497 every 8 connection(s)
I20260619 02:43:09.487897 21019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:09.488989 21087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:09.490644 21087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:09.498077 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21019
I20260619 02:43:09.498238 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:43:09.498450 7716 external_mini_cluster.cc:1428] Setting key 3d350ee759cfff68a692442ada2efe1f
I20260619 02:43:09.500475 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.491373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36497 (local address 127.0.0.1:47026)
0619 02:43:09.491556 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.491595 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.492052 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.492351 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.492354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.492618 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.492859 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.492864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.493840 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.493841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.494580 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.494582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.495067 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.495068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.495122 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.495603 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.495612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.497116 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.498919 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.498926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.498928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.499219 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.499454 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.499454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.499455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.499573 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.500052 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.500058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.500114 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.500191 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.500313 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260619 02:43:09.500566 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.491535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.7.137.62:36497)
0619 02:43:09.491666 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.491670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.491699 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:43:09.492207 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.492209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.492234 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.492320 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.492948 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.493748 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.494660 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.494991 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.495074 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.497190 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.497197 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.497197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.497203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.498843 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.499301 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.499304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.499304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.499383 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.499638 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.499638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.499639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.499919 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.500066 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.500252 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.500400 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260619 02:43:09.501343 21087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:43:09.501710 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.501578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45138)
0619 02:43:09.501630 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.501679 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:43:09.501860 21037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
I20260619 02:43:09.502344 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.502226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45148)
0619 02:43:09.502281 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.502313 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:43:09.502463 21087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:46747: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.137.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
I20260619 02:43:09.523682 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:46747
--webserver_interface=127.7.137.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:09.551726 21087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } attempt: 1
I20260619 02:43:09.552224 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.552086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45156)
0619 02:43:09.552156 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.552193 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:43:09.552645 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.552557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45162)
0619 02:43:09.552602 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.552627 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 02:43:09.552772 21087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:46747: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
W20260619 02:43:09.605460 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:09.605692 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:09.605731 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:09.606981 21094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:09.607044 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:09.607066 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:09.607085 21094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:09.607103 21094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:09.608636 21094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:46747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.137.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:09.608844 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:09.609011 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:09.613509 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:09.613507 21102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:09.613507 21100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:09.613842 21094 server_base.cc:1034] running on GCE node
I20260619 02:43:09.614106 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:09.614383 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:09.615545 21094 hybrid_clock.cc:648] HybridClock initialized: now 1781836989615525 us; error 39 us; skew 500 ppm
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:09.618223 21094 init.cc:377] Logged in from keytab as kudu/127.7.137.61@KRBTEST.COM (short username kudu)
I20260619 02:43:09.618613 21094 webserver.cc:460] Webserver started at http://127.7.137.61:42057/ using document root <none> and password file <none>
I20260619 02:43:09.618853 21094 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:09.618923 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:09.619092 21094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:09.620159 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data/instance:
uuid: "334794e80a844237a7e515040cf16c3a"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "8e78f39b86bbbd89ded2c89f3698b3dd"
server_key_iv: "61bb5d3b3f0528bbf48f6a515752f65c"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.620518 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal/instance:
uuid: "334794e80a844237a7e515040cf16c3a"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "8e78f39b86bbbd89ded2c89f3698b3dd"
server_key_iv: "61bb5d3b3f0528bbf48f6a515752f65c"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.620913 21087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } attempt: 2
I20260619 02:43:09.621326 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.621171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45172)
0619 02:43:09.621240 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.621281 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:43:09.621701 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.621603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45188)
0619 02:43:09.621653 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.621680 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:43:09.621835 21087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:46747: Network error: Client connection negotiation failed: client connection to 127.7.137.61:46747: connect: Connection refused (error 111)
I20260619 02:43:09.622200 21094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 02:43:09.622939 21109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:09.623119 21094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:09.623216 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "334794e80a844237a7e515040cf16c3a"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "8e78f39b86bbbd89ded2c89f3698b3dd"
server_key_iv: "61bb5d3b3f0528bbf48f6a515752f65c"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.623303 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:09.637538 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:09.639391 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:09.639550 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:09.643360 21094 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.61:46747
I20260619 02:43:09.643391 21161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.61:46747 every 8 connection(s)
I20260619 02:43:09.643733 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
I20260619 02:43:09.644574 21162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:09.646139 21162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:43:09.648440 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21094
I20260619 02:43:09.648602 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-1/wal/instance
I20260619 02:43:09.648826 7716 external_mini_cluster.cc:1428] Setting key a452d9b1ac9197a3f4f8e2b51cb299f7
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:09.656014 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.646869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.7.137.62:36497)
0619 02:43:09.646952 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.646954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.647163 (+ 209us) server_negotiation.cc:407] Connection header received
0619 02:43:09.647656 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.647658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.647675 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.647726 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.648339 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.648996 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.650046 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.650271 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.650332 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.653060 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.653061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.653061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.653067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.654501 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.655043 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.655044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.655044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.655096 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.655387 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.655387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.655388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.655484 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.655532 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.655884 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.655950 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:43:09.656306 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.646842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36497 (local address 127.0.0.1:47032)
0619 02:43:09.647031 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.647064 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.647534 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.647795 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.647799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.648041 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.648227 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.648230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.649098 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.649099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.649941 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.649944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.650355 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.650356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.650518 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.651051 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.651059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.652966 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.654619 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.654627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.654630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.654965 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.655165 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.655166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.655167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.655320 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.655555 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.655562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.655741 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.655876 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.656047 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260619 02:43:09.656906 21162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.137.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.60@KRBTEST.COM" created.
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:43:09.665098 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.657159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45190)
0619 02:43:09.657204 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.657215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.657254 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.657612 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.657613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.657730 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.657775 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.657776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.658750 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.658751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.659348 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.659351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.659935 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.659936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.659989 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.660357 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.660361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.661791 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.664045 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.664046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.664046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.664230 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.664467 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.664467 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.664468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.664515 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.664880 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.664881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.664914 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.664969 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.665035 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:43:09.665277 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.657232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45190 (local address 127.7.137.61:46747)
0619 02:43:09.657435 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.657441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.657465 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:43:09.657499 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.657501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.657517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.657611 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.657848 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.658663 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.659421 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.659861 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.659940 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.661882 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.661889 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.661889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.661895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.663960 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.664319 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.664322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.664323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.664392 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.664578 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.664579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.664579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.664808 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.664912 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.665011 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.665107 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260619 02:43:09.665910 21162 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:43:09.666286 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.666146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56056)
0619 02:43:09.666204 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.666259 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:43:09.666450 21110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
I20260619 02:43:09.666900 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.666807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56066)
0619 02:43:09.666858 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.666883 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 02:43:09.667024 21162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:35501: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
I20260619 02:43:09.676164 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:35501
--webserver_interface=127.7.137.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:09.712990 21087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } attempt: 3
I20260619 02:43:09.716260 21162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } attempt: 1
I20260619 02:43:09.716663 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.716537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56078)
0619 02:43:09.716594 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.716632 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:43:09.717118 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.716916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56084)
0619 02:43:09.717061 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.717088 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
W20260619 02:43:09.717226 21162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:35501: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:43:09.720595 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.713462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.7.137.61:46747)
0619 02:43:09.713541 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.713544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.713557 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:43:09.713593 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.713595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.713609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.713701 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.714019 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.714844 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.715502 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.715817 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.715889 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.718089 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.718090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.718090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.718096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.719525 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.719923 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.719924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.719924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.719973 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.720175 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.720176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.720176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.720275 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.720325 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.720479 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.720540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:43:09.720656 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.713356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45202)
0619 02:43:09.713420 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.713436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.713489 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.713706 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.713707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.713857 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.713923 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.713925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.714927 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.714927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.715415 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.715416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.715894 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.715894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.715952 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.716501 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.716507 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.718011 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.719613 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.719614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.719614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.719850 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.720039 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.720040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.720041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.720105 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.720335 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.720336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.720385 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.720454 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.720558 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:43:09.721061 21087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:43:09.721403 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.721296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56092)
0619 02:43:09.721345 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.721378 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:43:09.721776 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.721660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56102)
0619 02:43:09.721713 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.721745 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:43:09.721874 21087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:35501: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
I20260619 02:43:09.753033 21087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } attempt: 1
I20260619 02:43:09.753558 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.753365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56108)
0619 02:43:09.753439 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.753519 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 02:43:09.753943 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.753835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56114)
0619 02:43:09.753882 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.753913 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:43:09.754060 21087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:35501: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
W20260619 02:43:09.761490 21169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:09.761672 21169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:09.761693 21169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:09.762739 21169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:09.762771 21169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:09.762784 21169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:09.762797 21169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:09.762809 21169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:09.764174 21169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36497,127.7.137.61:46747,127.7.137.60:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.137.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:09.764353 21169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:09.764515 21169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:09.769161 21178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:09.769207 21175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:09.769161 21176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:09.769425 21169 server_base.cc:1034] running on GCE node
I20260619 02:43:09.769692 21169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:09.769982 21169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:09.771171 21169 hybrid_clock.cc:648] HybridClock initialized: now 1781836989771135 us; error 47 us; skew 500 ppm
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:09.773896 21169 init.cc:377] Logged in from keytab as kudu/127.7.137.60@KRBTEST.COM (short username kudu)
I20260619 02:43:09.774322 21169 webserver.cc:460] Webserver started at http://127.7.137.60:44777/ using document root <none> and password file <none>
I20260619 02:43:09.774559 21169 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:09.774641 21169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:09.774786 21169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:09.775830 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data/instance:
uuid: "3988d68053c04859a4f3120c72ba3889"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "fa9e4f5ccc4249ac5ca0c869c8dd722c"
server_key_iv: "a399caaa55e7143f8aa7079690bfb48b"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.776183 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal/instance:
uuid: "3988d68053c04859a4f3120c72ba3889"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "fa9e4f5ccc4249ac5ca0c869c8dd722c"
server_key_iv: "a399caaa55e7143f8aa7079690bfb48b"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.777561 21169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:09.778316 21185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:09.778492 21169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:09.778594 21169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "3988d68053c04859a4f3120c72ba3889"
format_stamp: "Formatted at 2026-06-19 02:43:09 on dist-test-slave-pdnb"
server_key: "fa9e4f5ccc4249ac5ca0c869c8dd722c"
server_key_iv: "a399caaa55e7143f8aa7079690bfb48b"
server_key_version: "encryptionkey@0"
I20260619 02:43:09.778679 21169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:09.785363 21162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } attempt: 2
I20260619 02:43:09.785750 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.785623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56122)
0619 02:43:09.785681 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.785717 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:43:09.786149 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.786049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56132)
0619 02:43:09.786096 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.786128 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:43:09.786290 21162 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:35501: Network error: Client connection negotiation failed: client connection to 127.7.137.60:35501: connect: Connection refused (error 111)
I20260619 02:43:09.786774 21169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:09.788478 21169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:09.788631 21169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:09.792384 21169 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.60:35501
I20260619 02:43:09.792454 21237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.60:35501 every 8 connection(s)
I20260619 02:43:09.792858 21169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
I20260619 02:43:09.793706 21238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:09.795174 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:09.800890 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21169
I20260619 02:43:09.801007 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836839601721-7716-0/minicluster-data/master-2/wal/instance
I20260619 02:43:09.801229 7716 external_mini_cluster.cc:1428] Setting key d0b46576e6686386768ae243e2f75806
I20260619 02:43:09.803958 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.795842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.7.137.62:36497)
0619 02:43:09.795909 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.795911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.796135 (+ 224us) server_negotiation.cc:407] Connection header received
0619 02:43:09.796535 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.796536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.796546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.796596 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.797247 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.797802 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.798732 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.798917 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.798977 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.801013 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.801015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.801016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.801024 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.802600 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.803038 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.803039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.803040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.803088 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.803327 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.803327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.803327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.803421 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.803469 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.803786 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.803866 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:09.804055 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.795824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36497 (local address 127.0.0.1:47042)
0619 02:43:09.796010 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.796042 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.796450 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.796676 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.796679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.796991 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.797173 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.797177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.797880 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.797881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.798662 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.798664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.798998 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.798999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.799101 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.799590 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.799598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.800901 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.802676 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.802681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.802683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.802953 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.803149 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.803150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.803150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.803268 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.803478 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.803483 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.803626 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.803738 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.803867 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260619 02:43:09.804533 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:43:09.811035 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.801674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.7.137.60:35501)
0619 02:43:09.801851 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.801856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.801881 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:43:09.801939 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.801941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.801953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.802014 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.802583 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.803656 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.804891 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.805373 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.805439 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.808057 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.808064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.808065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.808072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.809818 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810215 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.810220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.810220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.810276 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810494 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.810495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.810496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.810634 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.810793 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.810848 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.810918 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260619 02:43:09.811316 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.804800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:46747 (local address 127.0.0.1:45210)
0619 02:43:09.804914 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.804924 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.804967 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.805078 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.805079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.805215 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.805267 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.805268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.805827 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.805828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.806341 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.806343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.806840 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.806841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.806890 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.807238 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.807244 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.808577 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.810306 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.810308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.810471 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.810679 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.810680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.810735 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.810984 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.810986 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.811024 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.811107 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.811221 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 02:43:09.811530 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.804869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.7.137.61:46747)
0619 02:43:09.804916 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.804918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.804955 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:43:09.805022 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.805023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.805030 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.805068 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.805333 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.805756 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.806427 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.806758 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.806840 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.808695 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.808696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.808696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.808704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.810233 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810550 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.810551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.810551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.810603 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.810808 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.810809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.810809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.810912 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.810981 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.811236 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.811305 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:43:09.811847 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:09.818616 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.812077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56146)
0619 02:43:09.812172 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.812181 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.812228 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.812349 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.812350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.812453 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.812480 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.812481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.813184 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.813185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.813636 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.813637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.814011 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.814012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.814060 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.814395 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.814399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.815684 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.817454 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.817455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.817455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.817628 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.817824 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.817825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.817825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.817872 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.818160 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.818161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.818238 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.818387 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.818487 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 02:43:09.818701 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.812139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.7.137.60:35501)
0619 02:43:09.812174 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.812176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.812211 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:43:09.812268 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.812269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.812281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.812350 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.812567 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.813092 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.813712 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.813946 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.814017 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.815763 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.815764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.815765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.815771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.817375 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.817699 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.817700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.817700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.817755 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.817933 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.817934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.817934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.818066 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.818147 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.818406 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.818528 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":17}
I20260619 02:43:09.819571 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.811793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.7.137.62:36497)
0619 02:43:09.811839 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.811841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.811849 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:09.811872 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.811873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.811883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.811933 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.812540 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.813085 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.813777 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.813948 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.814024 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.816608 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.816609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.816610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.816617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.818145 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.818584 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.818585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.818586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.818648 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.818882 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.818883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.818883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.819011 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.819181 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.819400 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.819462 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:43:09.820446 21238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: Bootstrap starting.
I20260619 02:43:09.821501 21238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:09.821823 21238 log.cc:824] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:09.822636 21238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: No bootstrap required, opened a new log
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:43:09.824700 21238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.824892 21238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:09.824940 21238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3988d68053c04859a4f3120c72ba3889, State: Initialized, Role: FOLLOWER
I20260619 02:43:09.825065 21238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.825408 21238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:09.825390 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.825531 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:43:09.827539 21255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:09.827620 21255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:09.828311 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.820185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45218 (local address 127.7.137.61:46747)
0619 02:43:09.820225 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.820227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.820274 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:43:09.820335 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.820336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.820345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.820394 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.820883 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.821580 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.822452 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.822627 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.822685 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.825079 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.825080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.825081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.825088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.826960 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.827350 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.827351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.827351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.827416 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.827632 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.827633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.827633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.827757 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.827925 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.828126 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.828209 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:43:09.829201 21087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } attempt: 2
I20260619 02:43:09.829587 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:09.827868 21169 master_runner.cc:186] Error getting master registration for 127.7.137.62:36497: OK, instance_id { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" instance_seqno: 1781836989486203 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:43:09.830340 21169 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:43:09.840314 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.830988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.7.137.62:36497)
0619 02:43:09.831025 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.831027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.833092 (+ 2065us) server_negotiation.cc:407] Connection header received
0619 02:43:09.833194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.833195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.833205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.833234 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.833940 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.834640 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.835660 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.835939 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.835979 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.837443 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.837445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.837445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.837452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.839085 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.839485 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.839486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.839486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.839540 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.839732 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.839733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.839733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.839853 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.839921 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.840138 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.840216 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:43:09.842729 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.831147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.7.137.60:35501)
0619 02:43:09.831187 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.831189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.832870 (+ 1681us) server_negotiation.cc:407] Connection header received
0619 02:43:09.832915 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.832916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.832926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.832982 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.833687 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.834697 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.835733 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.836036 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.836077 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.839002 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.839004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.839004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.839011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.840651 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.841950 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.841952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.841952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.842007 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.842210 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.842210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.842211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.842324 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.842395 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.842568 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.842639 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:09.843981 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.829488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56160)
0619 02:43:09.829569 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.829580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.829615 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.831590 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.831591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.831713 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.831750 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.831751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.832705 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.832705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.833577 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.833581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.834166 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.834167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.834221 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.834638 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.834643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.836033 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.841631 (+ 5598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.841632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.841633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.841852 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.842374 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.842375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.842375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.842439 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.843633 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.843634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.843679 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.843760 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.843858 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 02:43:09.843992 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.831288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.7.137.60:35501)
0619 02:43:09.831461 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.831462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.831470 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:09.831489 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.831490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.831499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.831574 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.831824 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.832622 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.833771 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.834058 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.834117 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.836122 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.836123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.836124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.836130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.841541 (+ 5411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.842226 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.842227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.842227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.842289 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.843190 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.843192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.843192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.843550 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.843607 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.843800 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.843873 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260619 02:43:09.845175 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.831225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45228 (local address 127.7.137.61:46747)
0619 02:43:09.831283 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.831285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.833663 (+ 2378us) server_negotiation.cc:407] Connection header received
0619 02:43:09.833684 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.833685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.833694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.833740 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.838103 (+ 4363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.838870 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.840092 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.840381 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.840426 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.841738 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.841739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.841739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.841746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.843376 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.843749 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.843750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.843750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.843808 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.844014 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.844015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.844016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.844147 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.844751 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.844976 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.845055 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:09.846143 21087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: Bootstrap starting.
I20260619 02:43:09.846710 21087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:09.847007 21087 log.cc:824] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:09.847793 21087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: No bootstrap required, opened a new log
I20260619 02:43:09.849308 21087 raft_consensus.cc:348] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.849416 21087 raft_consensus.cc:374] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:09.849438 21087 raft_consensus.cc:729] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78e1ba5db4645a181a556d47e7cc26c, State: Initialized, Role: FOLLOWER
I20260619 02:43:09.849591 21087 consensus_queue.cc:260] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.849843 21266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.849920 21266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:43:09.849921 21087 sys_catalog.cc:564] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:43:09.852741 21277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:09.852833 21277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:09.852994 21019 master_runner.cc:186] Error getting master registration for 127.7.137.61:46747: OK, instance_id { permanent_uuid: "334794e80a844237a7e515040cf16c3a" instance_seqno: 1781836989642320 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:43:09.855315 21019 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:43:09.877473 21162 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } attempt: 3
Jun 19 02:43:09 dist-test-slave-pdnb krb5kdc[21006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:43:09.886415 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.877794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:35501 (local address 127.0.0.1:56180)
0619 02:43:09.877851 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:09.877863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:09.877901 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:09.878078 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:09.878079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:09.878276 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:09.878332 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.878334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.879595 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.879596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.880422 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:09.880424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.881160 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.881161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:09.881224 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.881809 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:09.881814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:09.883378 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:09.885267 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.885268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.885268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.885493 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.885708 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:09.885709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:09.885710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:09.885780 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:09.886075 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:09.886076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:09.886130 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 02:43:09.886204 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:43:09.886304 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:43:09.886433 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:09.877889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.7.137.60:35501)
0619 02:43:09.877932 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:09.877934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:09.877943 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:09.877970 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:09.877971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:09.877991 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:09.878059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:09.878416 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.879502 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.880702 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:09.881086 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:09.881159 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:09.883473 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:09.883475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:09.883475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:09.883483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:09.885183 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.885574 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.885575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.885576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.885636 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:09.885849 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:09.885850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:09.885850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:09.886003 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:09.886073 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:09.886241 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:43:09.886331 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:43:09.888549 21162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: Bootstrap starting.
I20260619 02:43:09.889321 21162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:09.889628 21162 log.cc:824] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:09.891063 21162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: No bootstrap required, opened a new log
I20260619 02:43:09.893221 21162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.893404 21162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:09.893483 21162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 334794e80a844237a7e515040cf16c3a, State: Initialized, Role: FOLLOWER
I20260619 02:43:09.893639 21162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.894024 21162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:09.893973 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.895526 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:43:09.895829 21291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:09.895936 21291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:09.898104 21094 master_runner.cc:263] No leader master found from master 334794e80a844237a7e515040cf16c3a
I20260619 02:43:09.898162 21094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:43:09.909525 21280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:09.909767 21280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.910442 21212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334794e80a844237a7e515040cf16c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3988d68053c04859a4f3120c72ba3889" is_pre_election: true
I20260619 02:43:09.910602 21212 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 334794e80a844237a7e515040cf16c3a in term 0.
I20260619 02:43:09.910918 21110 leader_election.cc:302] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 334794e80a844237a7e515040cf16c3a, 3988d68053c04859a4f3120c72ba3889; no voters:
I20260619 02:43:09.912410 21061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334794e80a844237a7e515040cf16c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78e1ba5db4645a181a556d47e7cc26c" is_pre_election: true
I20260619 02:43:09.912560 21061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 334794e80a844237a7e515040cf16c3a in term 0.
I20260619 02:43:09.912969 21280 leader_election.cc:288] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d78e1ba5db4645a181a556d47e7cc26c (127.7.137.62:36497), 3988d68053c04859a4f3120c72ba3889 (127.7.137.60:35501)
I20260619 02:43:09.913115 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:09.913228 21280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:09.913306 21280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:09.914306 21280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.914608 21061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334794e80a844237a7e515040cf16c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78e1ba5db4645a181a556d47e7cc26c"
I20260619 02:43:09.914692 21061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:09.915565 21061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 334794e80a844237a7e515040cf16c3a in term 1.
I20260619 02:43:09.915800 21113 leader_election.cc:302] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 334794e80a844237a7e515040cf16c3a, d78e1ba5db4645a181a556d47e7cc26c; no voters:
I20260619 02:43:09.916139 21212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334794e80a844237a7e515040cf16c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3988d68053c04859a4f3120c72ba3889"
I20260619 02:43:09.916224 21212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:09.917191 21212 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 334794e80a844237a7e515040cf16c3a in term 1.
I20260619 02:43:09.917439 21280 leader_election.cc:288] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [CANDIDATE]: Term 1 election: Requested vote from peers d78e1ba5db4645a181a556d47e7cc26c (127.7.137.62:36497), 3988d68053c04859a4f3120c72ba3889 (127.7.137.60:35501)
I20260619 02:43:09.917651 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:09.917739 21280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [term 1 LEADER]: Becoming Leader. State: Replica: 334794e80a844237a7e515040cf16c3a, State: Running, Role: LEADER
I20260619 02:43:09.917856 21280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } }
I20260619 02:43:09.918385 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 334794e80a844237a7e515040cf16c3a. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.918519 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:09.918794 21295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:09.919059 21295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:09.921284 21212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [term 1 FOLLOWER]: Refusing update from remote peer 334794e80a844237a7e515040cf16c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:09.921666 21293 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:09.923050 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 334794e80a844237a7e515040cf16c3a. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.923154 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:43:09.924278 21061 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [term 1 FOLLOWER]: Refusing update from remote peer 334794e80a844237a7e515040cf16c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:09.924585 21293 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:09.925920 21266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 334794e80a844237a7e515040cf16c3a. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.926021 21266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:43:09.927098 21293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.927222 21293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:09.927630 21266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.927711 21266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:43:09.928030 21296 mvcc.cc:205] Tried to move back new op lower bound from 7298404310714818560 to 7298404310704574464. Current Snapshot: MvccSnapshot[applied={T|T < 7298404310714818560}]
I20260619 02:43:09.928591 21299 mvcc.cc:205] Tried to move back new op lower bound from 7298404310714818560 to 7298404310704574464. Current Snapshot: MvccSnapshot[applied={T|T < 7298404310714818560}]
I20260619 02:43:09.929172 21295 catalog_manager.cc:1347] Generated new cluster ID: 17386a91331e4c26a262a10db0b5e3d7
I20260619 02:43:09.929219 21295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:09.938417 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.938509 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:43:09.939919 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "334794e80a844237a7e515040cf16c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78e1ba5db4645a181a556d47e7cc26c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36497 } } peers { permanent_uuid: "334794e80a844237a7e515040cf16c3a" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 46747 } } peers { permanent_uuid: "3988d68053c04859a4f3120c72ba3889" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 35501 } } }
I20260619 02:43:09.940012 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:09.941022 21295 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:09.941780 21295 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:09.954835 21295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 334794e80a844237a7e515040cf16c3a: Generated new TSK 0
I20260619 02:43:09.955135 21295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:10.828094 21255 catalog_manager.cc:1259] Loaded cluster ID: 17386a91331e4c26a262a10db0b5e3d7
I20260619 02:43:10.828173 21255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: loading cluster ID for follower catalog manager: success
I20260619 02:43:10.829062 21255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: acquiring CA information for follower catalog manager: success
I20260619 02:43:10.829356 21255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3988d68053c04859a4f3120c72ba3889: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:43:10.853415 21277 catalog_manager.cc:1259] Loaded cluster ID: 17386a91331e4c26a262a10db0b5e3d7
I20260619 02:43:10.853493 21277 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: loading cluster ID for follower catalog manager: success
I20260619 02:43:10.854344 21277 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: acquiring CA information for follower catalog manager: success
I20260619 02:43:10.854728 21277 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d78e1ba5db4645a181a556d47e7cc26c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:43:11.015419 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21019
I20260619 02:43:11.020519 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21094
I20260619 02:43:11.025277 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21169
2026-06-19T02:43:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2839 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:11 dist-test-slave-pdnb krb5kdc[21315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:11 dist-test-slave-pdnb krb5kdc[21315](info): set up 2 sockets
Jun 19 02:43:11 dist-test-slave-pdnb krb5kdc[21315](info): commencing operation
krb5kdc: starting...
W20260619 02:43:12.061094 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:43:12.176010 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45941
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:12.257812 21328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:12.258085 21328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:12.258121 21328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:12.259282 21328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:12.259348 21328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:12.259372 21328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:12.259394 21328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:12.259423 21328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:12.260879 21328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:12.261147 21328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:12.261368 21328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:12.266139 21334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:12.266201 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:12.266218 21336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:12.266321 21328 server_base.cc:1034] running on GCE node
I20260619 02:43:12.266674 21328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:12.266960 21328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:12.268132 21328 hybrid_clock.cc:648] HybridClock initialized: now 1781836992268109 us; error 41 us; skew 500 ppm
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836992, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:12.270990 21328 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:12.271430 21328 webserver.cc:460] Webserver started at http://127.7.137.62:37689/ using document root <none> and password file <none>
I20260619 02:43:12.271657 21328 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:12.271730 21328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:12.271880 21328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:12.273007 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "74020df63e02411c991a55747014b55d"
format_stamp: "Formatted at 2026-06-19 02:43:12 on dist-test-slave-pdnb"
server_key: "2d957580d54b0a76aa03b2f9fb89e179"
server_key_iv: "cbe984c3df81dba8c42df0a4273fb22b"
server_key_version: "encryptionkey@0"
I20260619 02:43:12.273375 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "74020df63e02411c991a55747014b55d"
format_stamp: "Formatted at 2026-06-19 02:43:12 on dist-test-slave-pdnb"
server_key: "2d957580d54b0a76aa03b2f9fb89e179"
server_key_iv: "cbe984c3df81dba8c42df0a4273fb22b"
server_key_version: "encryptionkey@0"
I20260619 02:43:12.274717 21328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:12.275564 21343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:12.275754 21328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:12.275863 21328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "74020df63e02411c991a55747014b55d"
format_stamp: "Formatted at 2026-06-19 02:43:12 on dist-test-slave-pdnb"
server_key: "2d957580d54b0a76aa03b2f9fb89e179"
server_key_iv: "cbe984c3df81dba8c42df0a4273fb22b"
server_key_version: "encryptionkey@0"
I20260619 02:43:12.275938 21328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:12.298102 21328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:12.300522 21328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:12.300669 21328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:12.304500 21328 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45941
I20260619 02:43:12.304575 21395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45941 every 8 connection(s)
I20260619 02:43:12.304940 21328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:12.305783 21396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:12.308030 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d: Bootstrap starting.
I20260619 02:43:12.308619 21396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:12.308954 21396 log.cc:824] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:12.310137 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d: No bootstrap required, opened a new log
I20260619 02:43:12.310906 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21328
I20260619 02:43:12.311017 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:43:12.311224 7716 external_mini_cluster.cc:1428] Setting key 07bf5faaff61205c802998d3d1a3cb53
I20260619 02:43:12.313107 21396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } }
I20260619 02:43:12.313517 21396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:12.313604 21396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74020df63e02411c991a55747014b55d, State: Initialized, Role: FOLLOWER
I20260619 02:43:12.313791 21396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } }
I20260619 02:43:12.313891 21396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:12.313926 21396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:12.314002 21396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:12.314992 21396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } }
I20260619 02:43:12.315122 21396 leader_election.cc:302] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74020df63e02411c991a55747014b55d; no voters:
I20260619 02:43:12.315289 21396 leader_election.cc:288] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:12.315376 21401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:12.315529 21401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [term 1 LEADER]: Becoming Leader. State: Replica: 74020df63e02411c991a55747014b55d, State: Running, Role: LEADER
I20260619 02:43:12.315550 21396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:12.315713 21401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } }
I20260619 02:43:12.316242 21401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74020df63e02411c991a55747014b55d. Latest consensus state: current_term: 1 leader_uuid: "74020df63e02411c991a55747014b55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } } }
I20260619 02:43:12.316391 21401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:12.316666 21410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:12.317041 21403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74020df63e02411c991a55747014b55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74020df63e02411c991a55747014b55d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45941 } } }
I20260619 02:43:12.317159 21410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:12.317176 21403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d [sys.catalog]: This master's current role is: LEADER
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:12.321730 21410 catalog_manager.cc:1347] Generated new cluster ID: e42104cf37c84da5bcc9f61e079f29b5
I20260619 02:43:12.321789 21410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:12.325435 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:12.311667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.7.137.62:45941)
0619 02:43:12.311819 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:12.311828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:12.312265 (+ 437us) server_negotiation.cc:407] Connection header received
0619 02:43:12.313456 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:12.313465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:12.313531 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:12.313676 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:12.314627 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:12.316097 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:12.317241 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:12.317979 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:12.318409 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:12.321588 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:12.321591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:12.321591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:12.321604 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:12.323575 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:12.323936 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:12.323939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:12.323939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:12.324026 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:12.324197 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:12.324198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:12.324198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:12.324468 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:12.325030 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:12.325103 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:43:12.325230 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260619 02:43:12.326229 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:43:12.360365 21410 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:12.361256 21410 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:12.365619 21410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74020df63e02411c991a55747014b55d: Generated new TSK 0
I20260619 02:43:12.365833 21410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:43:12.379987 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21328
2026-06-19T02:43:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1353 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21423](info): set up 2 sockets
Jun 19 02:43:12 dist-test-slave-pdnb krb5kdc[21423](info): commencing operation
krb5kdc: starting...
W20260619 02:43:13.416908 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:43:13.532207 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:13.615947 21436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.616158 21436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.616182 21436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.617352 21436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:13.617394 21436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.617411 21436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.617424 21436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:13.617439 21436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:13.618839 21436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:13.619056 21436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.619236 21436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:13.624148 21444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.624300 21436 server_base.cc:1034] running on GCE node
W20260619 02:43:13.624367 21441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.624131 21442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.624782 21436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.625120 21436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:13.626281 21436 hybrid_clock.cc:648] HybridClock initialized: now 1781836993626277 us; error 40 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:13.632890 21436 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:13.633353 21436 webserver.cc:460] Webserver started at http://127.7.137.62:33553/ using document root <none> and password file <none>
I20260619 02:43:13.633615 21436 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:13.633683 21436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:13.633827 21436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:13.634891 21436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "d1f5fd55f1a84b8f806ea69bef0106aa"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "194ba10ac691c48e25ecbd0ea647a87b"
server_key_iv: "40103bb574190dc7c3e624b11ded7b2f"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.635268 21436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "d1f5fd55f1a84b8f806ea69bef0106aa"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "194ba10ac691c48e25ecbd0ea647a87b"
server_key_iv: "40103bb574190dc7c3e624b11ded7b2f"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.636623 21436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:43:13.637452 21451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.637647 21436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:13.637742 21436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "d1f5fd55f1a84b8f806ea69bef0106aa"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "194ba10ac691c48e25ecbd0ea647a87b"
server_key_iv: "40103bb574190dc7c3e624b11ded7b2f"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.637831 21436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:13.650279 21436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:13.652361 21436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:13.652529 21436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:13.656426 21436 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41949
I20260619 02:43:13.656499 21503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41949 every 8 connection(s)
I20260619 02:43:13.656934 21436 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:13.657105 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21436
I20260619 02:43:13.657249 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:43:13.657531 7716 external_mini_cluster.cc:1428] Setting key 33618b20ecbbeea40fc697248c6d8251
I20260619 02:43:13.658388 21504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:13.661959 21504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap starting.
I20260619 02:43:13.662665 21504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:13.663009 21504 log.cc:824] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:13.663929 21504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: No bootstrap required, opened a new log
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:13.666392 21504 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:13.666574 21504 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:13.666649 21504 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Initialized, Role: FOLLOWER
I20260619 02:43:13.666819 21504 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:13.666932 21504 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:13.667011 21504 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:13.667085 21504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:13.667656 21504 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:13.667744 21504 leader_election.cc:302] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1f5fd55f1a84b8f806ea69bef0106aa; no voters:
I20260619 02:43:13.667945 21504 leader_election.cc:288] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:13.667972 21509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:13.668097 21509 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 LEADER]: Becoming Leader. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Running, Role: LEADER
I20260619 02:43:13.668180 21509 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:13.668222 21504 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:13.668525 21509 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f5fd55f1a84b8f806ea69bef0106aa. Latest consensus state: current_term: 1 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:13.668535 21510 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:13.668613 21509 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:13.668637 21510 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:13.669390 21514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:13.670014 21514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:13.670043 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.658167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.7.137.62:41949)
0619 02:43:13.658397 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:13.658407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:13.658446 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:43:13.659131 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:13.659142 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:13.659211 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:13.659397 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:13.659976 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.661613 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.662470 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.663173 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.663357 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:13.666068 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:13.666072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:13.666072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:13.666083 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:13.667957 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:13.668387 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:13.668390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:13.668390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:13.668492 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:13.668754 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:13.668755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:13.668755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:13.669082 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:13.669243 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:13.669670 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 02:43:13.669801 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260619 02:43:13.671708 21514 catalog_manager.cc:1347] Generated new cluster ID: f38b4f09743c4d67bafce2f0a9cd065c
I20260619 02:43:13.671759 21514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:13.686398 21514 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:13.687071 21514 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:13.690622 21514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Generated new TSK 0
I20260619 02:43:13.690819 21514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:43:13.722072 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:13.824064 21530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.824268 21530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.824292 21530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.825554 21530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.825595 21530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.825623 21530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:13.827072 21530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:13.827314 21530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.827591 21530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:13.832751 21535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.832764 21536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.832814 21538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.832926 21530 server_base.cc:1034] running on GCE node
I20260619 02:43:13.833237 21530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.833494 21530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:13.834677 21530 hybrid_clock.cc:648] HybridClock initialized: now 1781836993834654 us; error 49 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:13.837420 21530 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:13.837937 21530 webserver.cc:460] Webserver started at http://127.7.137.1:44535/ using document root <none> and password file <none>
I20260619 02:43:13.838181 21530 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:13.838246 21530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:13.838397 21530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:13.839536 21530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "4cba9cd890a94c0f8e009058a026a729"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "6fe329243b56f268eea3aee8680944a4"
server_key_iv: "a56d34ab603f5adadafc03c5b5f6c906"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.839887 21530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "4cba9cd890a94c0f8e009058a026a729"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "6fe329243b56f268eea3aee8680944a4"
server_key_iv: "a56d34ab603f5adadafc03c5b5f6c906"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.841351 21530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:13.842253 21545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.842506 21530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:13.842619 21530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "4cba9cd890a94c0f8e009058a026a729"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "6fe329243b56f268eea3aee8680944a4"
server_key_iv: "a56d34ab603f5adadafc03c5b5f6c906"
server_key_version: "encryptionkey@0"
I20260619 02:43:13.842710 21530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:13.859403 21530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:13.861521 21530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:13.861693 21530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:13.861945 21530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:13.862269 21530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:13.862322 21530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.862360 21530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:13.862404 21530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:13.868434 21530 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:41897
I20260619 02:43:13.868475 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:41897 every 8 connection(s)
I20260619 02:43:13.868897 21530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:13.877818 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.869684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:35451 (local address 127.7.137.62:41949)
0619 02:43:13.869750 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:13.869752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:13.870045 (+ 293us) server_negotiation.cc:407] Connection header received
0619 02:43:13.870470 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:13.870472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:13.870481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:13.870534 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:13.871441 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.871935 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.872914 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.873131 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.873190 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:13.875189 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:13.875190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:13.875191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:13.875197 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:13.876437 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:13.876905 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:13.876906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:13.876907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:13.876960 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:13.877201 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:13.877201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:13.877202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:13.877311 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:13.877364 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:13.877669 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 02:43:13.877754 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:43:13.878047 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:13.869678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:35451)
0619 02:43:13.869874 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:13.869912 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:13.870384 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:13.870600 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:13.870603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:13.871168 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:13.871364 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:13.871368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.872012 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.872013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:13.872831 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:13.872833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:13.873208 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:13.873209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:13.873304 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:13.873765 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:13.873773 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:13.875109 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:13.876558 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:13.876565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:13.876567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:13.876825 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:13.877041 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:13.877041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:13.877042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:13.877143 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:13.877389 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:13.877395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:13.877519 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:43:13.877611 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:43:13.877732 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260619 02:43:13.878351 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21530
I20260619 02:43:13.878468 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:13.878662 7716 external_mini_cluster.cc:1428] Setting key 45c9030e117cd842c48984c242236e8e
I20260619 02:43:13.878667 21659 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:13.878778 21659 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:13.879040 21659 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:13.879576 21468 ts_manager.cc:194] Registered new tserver with Master: 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897)
I20260619 02:43:13.880175 21468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:35451
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:43:13.903875 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:13.990121 21664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:13.990387 21664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:13.990430 21664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:13.991703 21664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:13.991777 21664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:13.991824 21664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:13.993445 21664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:13.993719 21664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:13.993950 21664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:43:13.999199 21664 server_base.cc:1034] running on GCE node
W20260619 02:43:13.999146 21670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.999163 21669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:13.999146 21672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:13.999580 21664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:13.999856 21664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:14.001020 21664 hybrid_clock.cc:648] HybridClock initialized: now 1781836994001012 us; error 38 us; skew 500 ppm
Jun 19 02:43:13 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:14.003490 21664 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:14.003952 21664 webserver.cc:460] Webserver started at http://127.7.137.2:44827/ using document root <none> and password file <none>
I20260619 02:43:14.004165 21664 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:14.004206 21664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:14.004295 21664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:14.005424 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "926e10f7656c4076a30653214f3cb514"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "cfaf30128b91ec943ee7dccb80d1fcf0"
server_key_iv: "7365932a1203912e3fe9e3df9e310f41"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.005723 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "926e10f7656c4076a30653214f3cb514"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "cfaf30128b91ec943ee7dccb80d1fcf0"
server_key_iv: "7365932a1203912e3fe9e3df9e310f41"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.006992 21664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:14.007746 21679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.007915 21664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:14.008002 21664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "926e10f7656c4076a30653214f3cb514"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "cfaf30128b91ec943ee7dccb80d1fcf0"
server_key_iv: "7365932a1203912e3fe9e3df9e310f41"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.008064 21664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:14.016896 21664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:14.018901 21664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:14.019013 21664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:14.019217 21664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:14.019536 21664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:14.019569 21664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.019593 21664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:14.019639 21664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.025491 21664 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:38953
I20260619 02:43:14.025563 21792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:38953 every 8 connection(s)
I20260619 02:43:14.025887 21664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:14.028793 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21664
I20260619 02:43:14.028895 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:14.029137 7716 external_mini_cluster.cc:1428] Setting key e5851a38a1bbc6be14cdf6e1aafbd6da
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:14.036304 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.026592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:34913 (local address 127.7.137.62:41949)
0619 02:43:14.026640 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.026642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.026872 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:43:14.027429 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.027430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.027442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.027506 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:14.028465 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.029144 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.030618 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.030799 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.030854 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.033133 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:14.033134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:14.033134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:14.033144 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:14.034921 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.035320 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.035322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.035322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.035389 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.035624 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.035625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.035626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.035771 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:14.035846 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.036106 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.036195 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:14.036368 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.026578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:34913)
0619 02:43:14.026753 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.026788 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.027319 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.027587 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.027591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.028166 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:14.028361 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.028366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.029280 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.029281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.030536 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.030539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.030883 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.030884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.031033 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.031634 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:14.031643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:14.033042 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:14.035004 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.035025 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.035027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.035244 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.035461 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.035461 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.035461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.035558 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.035839 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:14.035842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:14.035958 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.036057 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.036176 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260619 02:43:14.036911 21793 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:14.037027 21793 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:14.037237 21793 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:14.037662 21468 ts_manager.cc:194] Registered new tserver with Master: 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953)
I20260619 02:43:14.038057 21468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:34913
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:43:14.053793 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:14.134711 21798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:14.134920 21798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:14.134943 21798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:14.136500 21798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:14.136560 21798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:14.136616 21798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:14.138314 21798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:14.138504 21798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:14.138687 21798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:14.143676 21803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:14.143764 21798 server_base.cc:1034] running on GCE node
W20260619 02:43:14.143713 21806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.143704 21804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:14.144042 21798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:14.144286 21798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:14.145439 21798 hybrid_clock.cc:648] HybridClock initialized: now 1781836994145439 us; error 36 us; skew 500 ppm
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:14.147920 21798 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:14.148370 21798 webserver.cc:460] Webserver started at http://127.7.137.3:35317/ using document root <none> and password file <none>
I20260619 02:43:14.148595 21798 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:14.148663 21798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:14.148830 21798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:14.149896 21798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
format_stamp: "Formatted at 2026-06-19 02:43:14 on dist-test-slave-pdnb"
server_key: "28741d8b134988229df55253954105c3"
server_key_iv: "25a076e23279fa7db3e8c51adc7e4f25"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.150257 21798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
format_stamp: "Formatted at 2026-06-19 02:43:14 on dist-test-slave-pdnb"
server_key: "28741d8b134988229df55253954105c3"
server_key_iv: "25a076e23279fa7db3e8c51adc7e4f25"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.151595 21798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:43:14.152383 21813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.152539 21798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:14.152638 21798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
format_stamp: "Formatted at 2026-06-19 02:43:14 on dist-test-slave-pdnb"
server_key: "28741d8b134988229df55253954105c3"
server_key_iv: "25a076e23279fa7db3e8c51adc7e4f25"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.152720 21798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:14.170133 21798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:14.172462 21798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:14.172626 21798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:14.172875 21798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:14.173197 21798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:14.173231 21798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.173283 21798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:14.173310 21798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.178900 21798 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:38065
I20260619 02:43:14.179008 21926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:38065 every 8 connection(s)
I20260619 02:43:14.179293 21798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:14.188235 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.179985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:35909 (local address 127.7.137.62:41949)
0619 02:43:14.180026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.180028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.180309 (+ 281us) server_negotiation.cc:407] Connection header received
0619 02:43:14.180896 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.180897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.180909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.180965 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:14.181891 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.182389 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.183402 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.183605 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.183695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.185707 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:14.185708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:14.185709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:14.185716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:14.186894 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.187294 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.187295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.187296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.187357 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.187600 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.187601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.187602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.187716 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:14.187792 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.188056 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.188136 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:43:14.188429 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.179990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:35909)
0619 02:43:14.180159 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.180194 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.180786 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.181039 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.181050 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.181605 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:14.181792 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.181796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.182481 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.182482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.183293 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.183295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.183715 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.183716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.183839 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.184275 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:14.184283 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:14.185615 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:14.186986 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.186992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.186995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.187217 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.187424 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.187425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.187425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.187531 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.187782 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:14.187789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:14.187928 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.188036 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.188201 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260619 02:43:14.188894 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21798
I20260619 02:43:14.188989 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:14.189062 21927 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:14.189177 21927 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:14.189204 7716 external_mini_cluster.cc:1428] Setting key 025e37a13963a208b7df7879bf6b2fe9
I20260619 02:43:14.189353 21927 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:14.189778 21468 ts_manager.cc:194] Registered new tserver with Master: a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065)
I20260619 02:43:14.190127 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:14.190156 21468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:35909
I20260619 02:43:14.196969 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.191572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.7.137.62:41949)
0619 02:43:14.191607 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.191609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.191706 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:43:14.191755 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.191756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.191763 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.191810 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:14.192320 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.192842 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.193517 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.193695 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.193743 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.194830 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:14.194832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:14.194832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:14.194836 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:14.196007 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.196295 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.196296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.196296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.196343 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.196494 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.196495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.196495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.196632 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:14.196708 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.196851 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.196908 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:43:14.198523 21468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:14.200129 21468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:14.206831 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.7.137.1:41897)
0619 02:43:14.202910 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.202915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.202925 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:14.202966 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.202968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.202984 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.203087 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.203305 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204473 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.205892 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206375 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206478 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206544 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.206703 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.206748 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260619 02:43:14.206828 21940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.0.0.1:57162)
0619 02:43:14.202669 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.202711 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.202780 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.203166 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.203170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.203172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.203247 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.203249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204574 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.204575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.205830 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.205832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206480 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.206570 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206584 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.206684 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.206741 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":5,"thread_start_us":49,"threads_started":1}
I20260619 02:43:14.206997 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:38953 (local address 127.0.0.1:49492)
0619 02:43:14.202975 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.202984 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.203037 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.203359 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.203360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.203362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.203421 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.203423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204676 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.204676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.205608 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.205610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206620 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.206694 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206698 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.206749 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.206796 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":86,"threads_started":1}
I20260619 02:43:14.207005 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.7.137.2:38953)
0619 02:43:14.202931 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.202935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.203043 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:43:14.203133 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.203135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.203151 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.203222 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.203557 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204577 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.205690 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206401 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206518 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206582 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.206871 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.206912 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260619 02:43:14.207110 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:38065 (local address 127.0.0.1:48166)
0619 02:43:14.202879 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.202889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.202925 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.203423 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.203423 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.203426 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.203486 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.203487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204406 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.204407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.205749 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.205752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206660 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.206726 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206730 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.206774 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.206805 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260619 02:43:14.207104 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.202735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.7.137.3:38065)
0619 02:43:14.203195 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.203199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.203208 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:14.203244 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.203246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.203262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.203340 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.203567 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.204308 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.205833 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.206523 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.206583 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.206633 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.206844 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.206893 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":72,"threads_started":1}
I20260619 02:43:14.207832 21727 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e (DEFAULT_TABLE table=test-table [id=95894872298748cda794359a4fa68be8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:14.207880 21861 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e (DEFAULT_TABLE table=test-table [id=95894872298748cda794359a4fa68be8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:14.208261 21727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9fad0eddd648aab7cf21d0ea9d401e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:14.208410 21861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9fad0eddd648aab7cf21d0ea9d401e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:14.208468 21593 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e (DEFAULT_TABLE table=test-table [id=95894872298748cda794359a4fa68be8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:14.208781 21593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9fad0eddd648aab7cf21d0ea9d401e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:14.211030 21944 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Bootstrap starting.
I20260619 02:43:14.211647 21944 tablet_bootstrap.cc:654] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:14.211793 21945 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Bootstrap starting.
I20260619 02:43:14.212196 21946 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Bootstrap starting.
I20260619 02:43:14.212416 21944 log.cc:824] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:14.212555 21945 tablet_bootstrap.cc:654] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:14.212898 21946 tablet_bootstrap.cc:654] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:14.213174 21945 log.cc:824] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:14.213294 21944 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: No bootstrap required, opened a new log
I20260619 02:43:14.213378 21944 ts_tablet_manager.cc:1397] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:14.213474 21946 log.cc:824] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:14.214008 21945 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: No bootstrap required, opened a new log
I20260619 02:43:14.214088 21945 ts_tablet_manager.cc:1397] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:14.215026 21944 raft_consensus.cc:348] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.215179 21944 raft_consensus.cc:374] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:14.215204 21944 raft_consensus.cc:729] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926e10f7656c4076a30653214f3cb514, State: Initialized, Role: FOLLOWER
I20260619 02:43:14.215332 21944 consensus_queue.cc:260] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.215672 21944 ts_tablet_manager.cc:1428] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:43:14.215690 21793 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
I20260619 02:43:14.215567 21945 raft_consensus.cc:348] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.215749 21945 raft_consensus.cc:374] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:14.215797 21945 raft_consensus.cc:729] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a567e6ab6f1d4118bcc8d3b28270939e, State: Initialized, Role: FOLLOWER
I20260619 02:43:14.215906 21945 consensus_queue.cc:260] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.216183 21945 ts_tablet_manager.cc:1428] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:14.216207 21927 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
I20260619 02:43:14.216886 21946 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: No bootstrap required, opened a new log
I20260619 02:43:14.216955 21946 ts_tablet_manager.cc:1397] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 02:43:14.218426 21946 raft_consensus.cc:348] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.218593 21946 raft_consensus.cc:374] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:14.218641 21946 raft_consensus.cc:729] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cba9cd890a94c0f8e009058a026a729, State: Initialized, Role: FOLLOWER
I20260619 02:43:14.218747 21946 consensus_queue.cc:260] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.219020 21946 ts_tablet_manager.cc:1428] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:14.219102 21659 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
W20260619 02:43:14.276495 21794 tablet.cc:2369] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:14.369624 21660 tablet.cc:2369] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:14.429844 21928 tablet.cc:2369] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:14.557104 21951 raft_consensus.cc:482] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:14.557257 21951 raft_consensus.cc:504] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.557708 21951 leader_election.cc:288] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897), 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953)
I20260619 02:43:14.561560 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.557999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:41703 (local address 127.7.137.2:38953)
0619 02:43:14.558047 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.558049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.558060 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:14.558089 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.558090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.558101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.558163 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.558354 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.559145 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.560632 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.561198 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.561273 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.561284 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.561452 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.561488 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:43:14.561559 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.557989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:55923 (local address 127.7.137.1:41897)
0619 02:43:14.558062 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.558065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.558142 (+ 77us) server_negotiation.cc:407] Connection header received
0619 02:43:14.558216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.558217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.558228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.558281 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.558429 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.559146 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.560632 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.561198 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.561273 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.561283 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.561451 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.561488 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:43:14.561583 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.557852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:38953 (local address 127.7.137.3:41703)
0619 02:43:14.557917 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.557930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.557980 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.558181 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.558182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.558184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.558277 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.558280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.559232 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.559232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.560548 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.560550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.561282 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.561283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.561368 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.561370 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.561428 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.561485 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":92}
I20260619 02:43:14.561589 21953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.557863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.7.137.3:55923)
0619 02:43:14.558085 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.558101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.558164 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.558295 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.558296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.558297 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.558361 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.558362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.559232 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.559232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.560536 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.560539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.561282 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.561283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.561364 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.561368 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.561429 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.561485 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":84,"thread_start_us":91,"threads_started":1}
I20260619 02:43:14.561969 21747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9fad0eddd648aab7cf21d0ea9d401e" candidate_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926e10f7656c4076a30653214f3cb514" is_pre_election: true
I20260619 02:43:14.562143 21747 raft_consensus.cc:2454] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a567e6ab6f1d4118bcc8d3b28270939e in term 0.
I20260619 02:43:14.562093 21613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9fad0eddd648aab7cf21d0ea9d401e" candidate_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cba9cd890a94c0f8e009058a026a729" is_pre_election: true
I20260619 02:43:14.562260 21613 raft_consensus.cc:2454] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a567e6ab6f1d4118bcc8d3b28270939e in term 0.
I20260619 02:43:14.562369 21817 leader_election.cc:302] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 926e10f7656c4076a30653214f3cb514, a567e6ab6f1d4118bcc8d3b28270939e; no voters:
I20260619 02:43:14.562495 21951 raft_consensus.cc:2781] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:14.562556 21951 raft_consensus.cc:482] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:14.562579 21951 raft_consensus.cc:3037] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:14.563256 21951 raft_consensus.cc:504] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.563387 21951 leader_election.cc:288] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [CANDIDATE]: Term 1 election: Requested vote from peers 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897), 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953)
I20260619 02:43:14.563565 21613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9fad0eddd648aab7cf21d0ea9d401e" candidate_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cba9cd890a94c0f8e009058a026a729"
I20260619 02:43:14.563565 21747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9fad0eddd648aab7cf21d0ea9d401e" candidate_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926e10f7656c4076a30653214f3cb514"
I20260619 02:43:14.563649 21747 raft_consensus.cc:3037] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:14.563650 21613 raft_consensus.cc:3037] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:14.564534 21747 raft_consensus.cc:2454] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a567e6ab6f1d4118bcc8d3b28270939e in term 1.
I20260619 02:43:14.564549 21613 raft_consensus.cc:2454] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a567e6ab6f1d4118bcc8d3b28270939e in term 1.
I20260619 02:43:14.564710 21816 leader_election.cc:302] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cba9cd890a94c0f8e009058a026a729, a567e6ab6f1d4118bcc8d3b28270939e; no voters:
I20260619 02:43:14.564841 21951 raft_consensus.cc:2781] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:14.565018 21951 raft_consensus.cc:686] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 1 LEADER]: Becoming Leader. State: Replica: a567e6ab6f1d4118bcc8d3b28270939e, State: Running, Role: LEADER
I20260619 02:43:14.565136 21951 consensus_queue.cc:237] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.565929 21468 catalog_manager.cc:5495] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e reported cstate change: term changed from 0 to 1, leader changed from <none> to a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3). New cstate: current_term: 1 leader_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } health_report { overall_health: HEALTHY } } }
I20260619 02:43:14.649545 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.647262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.7.137.3:38065)
0619 02:43:14.647324 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.647326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.647332 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:43:14.647368 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.647369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.647389 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.647444 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:14.647589 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.648067 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.648819 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.648997 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.649061 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.649205 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:14.649305 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:14.649364 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.649401 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.649417 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:43:14.651398 21613 raft_consensus.cc:1261] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Refusing update from remote peer a567e6ab6f1d4118bcc8d3b28270939e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:14.651401 21747 raft_consensus.cc:1261] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Refusing update from remote peer a567e6ab6f1d4118bcc8d3b28270939e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:14.651779 21954 consensus_queue.cc:1035] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:14.651917 21951 consensus_queue.cc:1035] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [LEADER]: Connected to new peer: Peer: permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:14.654021 21957 mvcc.cc:205] Tried to move back new op lower bound from 7298404330089230336 to 7298404329739374592. Current Snapshot: MvccSnapshot[applied={T|T < 7298404330089230336}]
I20260619 02:43:14.654573 21959 mvcc.cc:205] Tried to move back new op lower bound from 7298404330089230336 to 7298404329739374592. Current Snapshot: MvccSnapshot[applied={T|T < 7298404330089230336}]
I20260619 02:43:14.655169 21958 mvcc.cc:205] Tried to move back new op lower bound from 7298404330089230336 to 7298404329739374592. Current Snapshot: MvccSnapshot[applied={T|T < 7298404330089230336 or (T in {7298404330089230336})}]
I20260619 02:43:14.656277 21467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:14.656397 21467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:14.657176 21467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Sending DeleteTablet for 3 replicas of tablet 3d9fad0eddd648aab7cf21d0ea9d401e
I20260619 02:43:14.657509 21593 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:14 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:57162
I20260619 02:43:14.657544 21727 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:14 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:49492
I20260619 02:43:14.657897 21967 tablet_replica.cc:330] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: stopping tablet replica
I20260619 02:43:14.658008 21967 raft_consensus.cc:2229] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:14.658124 21967 raft_consensus.cc:2258] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:14.658279 21966 tablet_replica.cc:330] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: stopping tablet replica
I20260619 02:43:14.658368 21966 raft_consensus.cc:2229] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:14.658470 21966 raft_consensus.cc:2258] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:14.658893 21967 ts_tablet_manager.cc:1905] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:14.658898 21966 ts_tablet_manager.cc:1905] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:14.659282 21861 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:14 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:48166
I20260619 02:43:14.659502 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21530
I20260619 02:43:14.660811 21967 ts_tablet_manager.cc:1918] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:14.660885 21967 log.cc:1196] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/3d9fad0eddd648aab7cf21d0ea9d401e
I20260619 02:43:14.660979 21968 tablet_replica.cc:330] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: stopping tablet replica
I20260619 02:43:14.661070 21968 raft_consensus.cc:2229] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:14.661157 21967 ts_tablet_manager.cc:1939] T 3d9fad0eddd648aab7cf21d0ea9d401e P 926e10f7656c4076a30653214f3cb514: Deleting consensus metadata
I20260619 02:43:14.661186 21968 raft_consensus.cc:2258] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:14.661545 21968 ts_tablet_manager.cc:1905] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:14.661650 21455 catalog_manager.cc:4841] TS 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953): tablet 3d9fad0eddd648aab7cf21d0ea9d401e (table test-table [id=95894872298748cda794359a4fa68be8]) successfully deleted
I20260619 02:43:14.663414 21968 ts_tablet_manager.cc:1918] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:14.663494 21968 log.cc:1196] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/3d9fad0eddd648aab7cf21d0ea9d401e
I20260619 02:43:14.663744 21968 ts_tablet_manager.cc:1939] T 3d9fad0eddd648aab7cf21d0ea9d401e P a567e6ab6f1d4118bcc8d3b28270939e: Deleting consensus metadata
I20260619 02:43:14.664127 21452 catalog_manager.cc:4841] TS a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065): tablet 3d9fad0eddd648aab7cf21d0ea9d401e (table test-table [id=95894872298748cda794359a4fa68be8]) successfully deleted
W20260619 02:43:14.665894 21454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:41897 (error 108)
I20260619 02:43:14.665949 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21664
I20260619 02:43:14.666978 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.666796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.0.0.1:57176)
0619 02:43:14.666888 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.666937 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260619 02:43:14.667124 21454 catalog_manager.cc:4584] TS 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d9fad0eddd648aab7cf21d0ea9d401e: Network error: Client connection negotiation failed: client connection to 127.7.137.1:41897: connect: Connection refused (error 111)
I20260619 02:43:14.671371 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21798
I20260619 02:43:14.675865 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21436
I20260619 02:43:14.680711 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--webserver_interface=127.7.137.62
--webserver_port=33553
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:14.762446 21970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:14.762643 21970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:14.762666 21970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:14.763782 21970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:14.763821 21970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:14.763834 21970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:14.763847 21970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:14.763859 21970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:14.765343 21970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=33553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:14.765573 21970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:14.765799 21970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:43:14.774252 21970 server_base.cc:1034] running on GCE node
W20260619 02:43:14.774204 21977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.774207 21976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.774204 21979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:14.774742 21970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:14.775053 21970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:14.776212 21970 hybrid_clock.cc:648] HybridClock initialized: now 1781836994776211 us; error 35 us; skew 500 ppm
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:14.778646 21970 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:14.779136 21970 webserver.cc:460] Webserver started at http://127.7.137.62:33553/ using document root <none> and password file <none>
I20260619 02:43:14.779368 21970 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:14.779448 21970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:14.780848 21970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:14.781492 21986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.781661 21970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:14.781757 21970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "d1f5fd55f1a84b8f806ea69bef0106aa"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "194ba10ac691c48e25ecbd0ea647a87b"
server_key_iv: "40103bb574190dc7c3e624b11ded7b2f"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.782160 21970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:14.793982 21970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:14.795663 21970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:14.795816 21970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:14.799952 21970 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41949
I20260619 02:43:14.799983 22038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41949 every 8 connection(s)
I20260619 02:43:14.800354 21970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:14.802016 22039 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:43:14.802680 22039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap starting.
I20260619 02:43:14.805433 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 21970
I20260619 02:43:14.805765 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:41897
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=44535
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:14.812258 22039 log.cc:824] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:14.814379 22039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:14.814821 22039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap complete.
I20260619 02:43:14.817276 22039 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:14.817513 22039 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Initialized, Role: FOLLOWER
I20260619 02:43:14.817682 22039 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:14.817745 22039 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:14.817797 22039 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:14.817847 22039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:43:14.818936 22039 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:14.819057 22039 leader_election.cc:302] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1f5fd55f1a84b8f806ea69bef0106aa; no voters:
I20260619 02:43:14.819257 22039 leader_election.cc:288] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:43:14.819322 22043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:43:14.819454 22043 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 LEADER]: Becoming Leader. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Running, Role: LEADER
I20260619 02:43:14.819537 22039 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:14.819555 22043 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:14.819816 22043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f5fd55f1a84b8f806ea69bef0106aa. Latest consensus state: current_term: 2 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:14.819886 22043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:14.820070 22048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:14.819819 22044 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:14.820219 22044 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:14.820884 22048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:14.820986 22048 catalog_manager.cc:1259] Loaded cluster ID: f38b4f09743c4d67bafce2f0a9cd065c
I20260619 02:43:14.821029 22048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:14.821286 22048 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 02:43:14.826519 22048 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 02:43:14.828913 22048 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:14.829107 22048 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Loaded TSK: 0
I20260619 02:43:14.829314 22048 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:43:14.890318 22041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:14.890522 22041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:14.890547 22041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:14.891786 22041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:14.891829 22041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:14.891856 22041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:14.893368 22041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:41897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=44535
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:14.893580 22041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:14.893824 22041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:14.899603 22069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.899557 22067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:14.899645 22066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:14.899981 22041 server_base.cc:1034] running on GCE node
I20260619 02:43:14.900225 22041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:14.900475 22041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:14.901628 22041 hybrid_clock.cc:648] HybridClock initialized: now 1781836994901611 us; error 32 us; skew 500 ppm
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:14.904043 22041 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:14.904444 22041 webserver.cc:460] Webserver started at http://127.7.137.1:44535/ using document root <none> and password file <none>
I20260619 02:43:14.904665 22041 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:14.904767 22041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:14.906113 22041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:14.906919 22076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:14.907138 22041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:14.907238 22041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "4cba9cd890a94c0f8e009058a026a729"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "6fe329243b56f268eea3aee8680944a4"
server_key_iv: "a56d34ab603f5adadafc03c5b5f6c906"
server_key_version: "encryptionkey@0"
I20260619 02:43:14.907697 22041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:14.923607 22041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:14.926069 22041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:14.926219 22041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:14.926456 22041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:14.926925 22083 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:43:14.928326 22041 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:43:14.928403 22041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:43:14.928439 22041 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:43:14.929335 22041 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:43:14.929382 22041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:14.929450 22083 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Bootstrap starting.
I20260619 02:43:14.930667 22083 log.cc:824] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:14.933537 22083 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:14.933841 22083 tablet_bootstrap.cc:492] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Bootstrap complete.
I20260619 02:43:14.933987 22083 ts_tablet_manager.cc:1397] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 02:43:14.935808 22083 raft_consensus.cc:348] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.936149 22083 raft_consensus.cc:729] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cba9cd890a94c0f8e009058a026a729, State: Initialized, Role: FOLLOWER
I20260619 02:43:14.936326 22083 consensus_queue.cc:260] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } } peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } }
I20260619 02:43:14.936605 22083 ts_tablet_manager.cc:1428] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:43:14.936764 22041 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:41897
I20260619 02:43:14.937107 22192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:41897 every 8 connection(s)
I20260619 02:43:14.937152 22041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:14.941085 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22041
I20260619 02:43:14.941381 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:38953
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=44827
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:43:14 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:14.950949 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.937967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:51633)
0619 02:43:14.938153 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.938198 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.938330 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.938733 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.938735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.939003 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:14.939187 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.939191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.940925 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.940926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.942215 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.942218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.943146 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.943147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.943281 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.943837 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:14.943850 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:14.946221 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:14.948821 (+ 2600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.948829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.948831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.949276 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.949634 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:14.949635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:14.949635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:14.949766 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:14.950253 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:14.950259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:14.950408 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.950517 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.950678 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260619 02:43:14.951013 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.938044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:51633 (local address 127.7.137.62:41949)
0619 02:43:14.938242 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.938251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.938295 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:43:14.938506 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.938513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.938568 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.938708 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:14.939329 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.940786 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.942314 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.942929 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.943050 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.946340 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:14.946345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:14.946345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:14.946358 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:14.948692 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.949400 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.949403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.949404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.949542 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:14.949848 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:14.949849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:14.949849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:14.950170 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:14.950266 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.950602 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.950756 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260619 02:43:14.952252 22193 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:14.952354 22193 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:14.952526 22193 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:14.953327 22003 ts_manager.cc:194] Registered new tserver with Master: 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897)
I20260619 02:43:14.954517 22003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:51633
I20260619 02:43:14.964490 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.954156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.7.137.1:41897)
0619 02:43:14.960860 (+ 6704us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:14.960864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:14.960883 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:43:14.960918 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:14.960920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:14.960935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:14.961015 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:14.961200 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.962163 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.963632 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.964121 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.964196 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.964233 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:14.964395 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:43:14.964427 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6669,"thread_start_us":70,"threads_started":1}
I20260619 02:43:14.964511 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:14.953933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.0.0.1:57182)
0619 02:43:14.956837 (+ 2904us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:14.956880 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:14.956945 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:14.961044 (+ 4099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:14.961048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:14.961050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:14.961117 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.961119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.962254 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.962255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.963531 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:14.963534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:14.964213 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:14.964214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:14.964291 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:14.964304 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:43:14.964376 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:43:14.964428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2873,"thread_start_us":80,"threads_started":1}
I20260619 02:43:14.964983 22126 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9fad0eddd648aab7cf21d0ea9d401e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:14 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:57182
I20260619 02:43:14.965674 22200 tablet_replica.cc:330] stopping tablet replica
I20260619 02:43:14.965797 22200 raft_consensus.cc:2229] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:14.965879 22200 raft_consensus.cc:2258] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:14.966163 22200 ts_tablet_manager.cc:1905] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:14.968137 22200 ts_tablet_manager.cc:1918] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:14.968222 22200 log.cc:1196] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/3d9fad0eddd648aab7cf21d0ea9d401e
I20260619 02:43:14.968437 22200 ts_tablet_manager.cc:1939] T 3d9fad0eddd648aab7cf21d0ea9d401e P 4cba9cd890a94c0f8e009058a026a729: Deleting consensus metadata
I20260619 02:43:14.968977 21989 catalog_manager.cc:4841] TS 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897): tablet 3d9fad0eddd648aab7cf21d0ea9d401e (table test-table [id=95894872298748cda794359a4fa68be8]) successfully deleted
W20260619 02:43:15.027552 22197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.027748 22197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.027773 22197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.028882 22197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.028918 22197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.028949 22197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:15.030309 22197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:38953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=44827
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.030491 22197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.030658 22197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.035599 22205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.035647 22206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.035663 22208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.035670 22197 server_base.cc:1034] running on GCE node
I20260619 02:43:15.036108 22197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.036347 22197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.037505 22197 hybrid_clock.cc:648] HybridClock initialized: now 1781836995037474 us; error 50 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.040005 22197 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:15.040428 22197 webserver.cc:460] Webserver started at http://127.7.137.2:44827/ using document root <none> and password file <none>
I20260619 02:43:15.040668 22197 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.040771 22197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.042151 22197 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:15.042791 22215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.043011 22197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:15.043119 22197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "926e10f7656c4076a30653214f3cb514"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "cfaf30128b91ec943ee7dccb80d1fcf0"
server_key_iv: "7365932a1203912e3fe9e3df9e310f41"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.043465 22197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.072918 22197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.075099 22197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.075280 22197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.075523 22197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:15.075837 22197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:15.075892 22197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.075937 22197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:15.075966 22197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.081772 22197 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:38953
I20260619 02:43:15.081816 22328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:38953 every 8 connection(s)
I20260619 02:43:15.082167 22197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:15.086769 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22197
I20260619 02:43:15.087085 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:38065
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=35317
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:15.092185 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.082848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:51691 (local address 127.7.137.62:41949)
0619 02:43:15.082896 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.082898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.083094 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:43:15.083599 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.083600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.083612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.083672 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:15.084673 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.085236 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.086294 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.086488 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.086550 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.088961 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:15.088963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:15.088963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:15.088970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:15.090409 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.091020 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.091021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.091022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.091102 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.091332 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.091333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.091334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.091500 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:15.091615 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.092030 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.092118 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:43:15.092790 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.082811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:51691)
0619 02:43:15.082973 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.083014 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.083510 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.083752 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.083756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.084319 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:15.084588 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.084594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.085363 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.085364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.086217 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.086220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.086567 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.086568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.086672 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.087224 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:15.087233 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:15.088864 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:15.090658 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.090666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.090668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.090925 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.091179 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.091179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.091179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.091268 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.091650 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:15.091657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:15.091846 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.091979 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.092184 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260619 02:43:15.093364 22329 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:15.093466 22329 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:15.093643 22329 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:15.094044 22003 ts_manager.cc:194] Registered new tserver with Master: 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953)
I20260619 02:43:15.094426 22003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:51691
W20260619 02:43:15.168790 22332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.169008 22332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.169045 22332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.170172 22332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.170236 22332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.170281 22332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:15.171881 22332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:38065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=35317
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.172144 22332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.172358 22332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.177395 22338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.177429 22337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.177431 22332 server_base.cc:1034] running on GCE node
W20260619 02:43:15.177402 22340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.177760 22332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.178004 22332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.179190 22332 hybrid_clock.cc:648] HybridClock initialized: now 1781836995179180 us; error 41 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.181929 22332 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:15.182415 22332 webserver.cc:460] Webserver started at http://127.7.137.3:35317/ using document root <none> and password file <none>
I20260619 02:43:15.182710 22332 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.182804 22332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.184271 22332 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:15.185048 22347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.185220 22332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.185317 22332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
format_stamp: "Formatted at 2026-06-19 02:43:14 on dist-test-slave-pdnb"
server_key: "28741d8b134988229df55253954105c3"
server_key_iv: "25a076e23279fa7db3e8c51adc7e4f25"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.185720 22332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.195163 22332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.196918 22332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.197067 22332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.197273 22332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:15.197561 22332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:15.197613 22332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.197651 22332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:15.197678 22332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.203518 22332 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:38065
I20260619 02:43:15.203558 22460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:38065 every 8 connection(s)
I20260619 02:43:15.203902 22332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:15.212261 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22332
I20260619 02:43:15.213351 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.204625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:40483 (local address 127.7.137.62:41949)
0619 02:43:15.204692 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.204695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.204940 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:43:15.205424 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.205425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.205435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.205494 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:15.206417 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.206940 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.207941 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.208147 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.208213 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.210340 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:15.210343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:15.210343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:15.210352 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:15.211813 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.212256 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.212257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.212257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.212309 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.212652 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.212653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.212654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.212833 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:15.212905 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.213168 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.213262 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:43:15.213541 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.204607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:40483)
0619 02:43:15.204808 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.204848 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.205324 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.205564 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.205568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.206118 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:15.206305 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.206309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.207020 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.207021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.207843 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.207845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.208226 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.208227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.208328 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.208899 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:15.208908 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:15.210233 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:15.211879 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.211884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.211886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.212183 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.212401 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:15.212402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:15.212402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:15.212544 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:15.212894 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:15.212900 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:15.213032 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.213144 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.213295 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260619 02:43:15.214004 22461 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41949
I20260619 02:43:15.214102 22461 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:15.214290 22461 heartbeater.cc:511] Master 127.7.137.62:41949 requested a full tablet report, sending...
I20260619 02:43:15.214660 22003 ts_manager.cc:194] Registered new tserver with Master: a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065)
I20260619 02:43:15.215042 22003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:40483
I20260619 02:43:15.219749 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.212710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.7.137.62:41949)
0619 02:43:15.212840 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.212843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.213021 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:43:15.213089 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.213090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.213104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.213186 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:15.213916 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.214550 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.215413 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.215630 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.215713 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.216955 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:15.216956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:15.216957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:15.216963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:15.218096 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.218434 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.218435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.218435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.218495 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:15.218700 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:15.218701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:15.218701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:15.218813 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:15.219509 (+ 696us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.219566 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.219645 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260619 02:43:15.220520 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:15.223343 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.220864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.7.137.62:41949)
0619 02:43:15.220934 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.220936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.221013 (+ 77us) server_negotiation.cc:407] Connection header received
0619 02:43:15.221101 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.221102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.221138 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.221216 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:15.221359 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.221819 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.222575 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.222788 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.222870 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.223041 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:15.223140 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:15.223233 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.223252 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.223272 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:43:15.223628 22003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:15.225291 22003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:15.227404 22126 tablet_service.cc:1467] Processing CreateTablet for tablet 6626c9b76973439a9904c6ad3a135530 (DEFAULT_TABLE table=test-table [id=ab52927938c14e6da6da8c742748a38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:15.227608 22126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6626c9b76973439a9904c6ad3a135530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:15.229061 22083 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Bootstrap starting.
I20260619 02:43:15.229527 22083 tablet_bootstrap.cc:654] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:15.230252 22083 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: No bootstrap required, opened a new log
I20260619 02:43:15.230304 22083 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.230463 22083 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.230538 22083 raft_consensus.cc:374] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:15.230587 22083 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cba9cd890a94c0f8e009058a026a729, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.230633 22083 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.230724 22083 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:15.230929 22193 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
I20260619 02:43:15.231040 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.227147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:38953 (local address 127.0.0.1:49508)
0619 02:43:15.227321 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.227332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.227406 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.227679 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.227680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.227682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:15.227722 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.227723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.228903 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.228904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.229796 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.229797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.230807 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.230808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.230865 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.230869 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.230915 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.230954 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260619 02:43:15.231069 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.227288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.7.137.2:38953)
0619 02:43:15.227518 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.227524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.227533 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:15.227571 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.227573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.227588 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.227684 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:15.227779 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.228791 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.229965 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.230722 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.230800 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.230911 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.230940 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.230983 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260619 02:43:15.231678 22263 tablet_service.cc:1467] Processing CreateTablet for tablet 6626c9b76973439a9904c6ad3a135530 (DEFAULT_TABLE table=test-table [id=ab52927938c14e6da6da8c742748a38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:15.231997 22263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6626c9b76973439a9904c6ad3a135530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:15.232761 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.227175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:38065 (local address 127.0.0.1:48192)
0619 02:43:15.229116 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.229128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.229186 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.229421 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.229422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.229424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:15.229458 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.229460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.230665 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.230666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.231691 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.231693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.232481 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.232482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.232547 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.232551 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.232615 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.232654 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1923,"thread_start_us":99,"threads_started":1}
I20260619 02:43:15.232822 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.227310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.7.137.3:38065)
0619 02:43:15.227522 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.227526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.229258 (+ 1732us) server_negotiation.cc:407] Connection header received
0619 02:43:15.229307 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.229310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.229329 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.229409 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:15.229529 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.230549 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.231763 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.232394 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.232451 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.232485 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.232667 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.232696 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
I20260619 02:43:15.233469 22395 tablet_service.cc:1467] Processing CreateTablet for tablet 6626c9b76973439a9904c6ad3a135530 (DEFAULT_TABLE table=test-table [id=ab52927938c14e6da6da8c742748a38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:15.233731 22395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6626c9b76973439a9904c6ad3a135530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:15.235036 22471 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Bootstrap starting.
I20260619 02:43:15.235760 22471 tablet_bootstrap.cc:654] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:15.236240 22471 log.cc:824] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:15.236397 22472 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Bootstrap starting.
I20260619 02:43:15.236991 22472 tablet_bootstrap.cc:654] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:15.236999 22471 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: No bootstrap required, opened a new log
I20260619 02:43:15.237072 22471 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:15.237473 22472 log.cc:824] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:15.238269 22472 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: No bootstrap required, opened a new log
I20260619 02:43:15.238358 22472 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:15.238870 22471 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.239056 22471 raft_consensus.cc:374] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:15.239101 22471 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926e10f7656c4076a30653214f3cb514, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.239223 22471 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.239506 22329 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
I20260619 02:43:15.239768 22471 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:15.239970 22472 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.240142 22472 raft_consensus.cc:374] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:15.240186 22472 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a567e6ab6f1d4118bcc8d3b28270939e, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.240299 22472 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.240571 22472 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:43:15.240654 22461 heartbeater.cc:503] Master 127.7.137.62:41949 was elected leader, sending a full tablet report...
I20260619 02:43:15.242486 22475 raft_consensus.cc:482] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:15.242592 22475 raft_consensus.cc:504] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.242970 22475 leader_election.cc:288] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065), 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897)
I20260619 02:43:15.246769 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.243302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:39551 (local address 127.7.137.3:38065)
0619 02:43:15.243362 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.243364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.243373 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:15.243393 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.243394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.243404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.243456 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:15.243592 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.244347 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.245850 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.246398 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.246475 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.246485 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.246683 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.246717 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:43:15.246767 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.243226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:59069 (local address 127.7.137.1:41897)
0619 02:43:15.243266 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.243268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.243276 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:15.243299 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.243300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.243317 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.243399 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:15.243589 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.244348 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.245795 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.246404 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.246475 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.246485 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.246659 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.246702 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:43:15.246764 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.243103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.7.137.2:59069)
0619 02:43:15.243159 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.243173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.243215 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.243410 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.243411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.243412 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:15.243497 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.243499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.244432 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.244432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.245704 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.245706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.246483 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.246484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.246582 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.246583 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.246641 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.246687 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":20}
I20260619 02:43:15.246773 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.243104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:38065 (local address 127.7.137.2:39551)
0619 02:43:15.243302 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.243310 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.243345 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.243469 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:15.243470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:15.243472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:15.243529 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.243530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.244432 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.244432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.245713 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:15.245715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.246483 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.246484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:15.246579 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.246583 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:15.246672 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:43:15.246699 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":103,"thread_start_us":80,"threads_started":1}
I20260619 02:43:15.247224 22145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "926e10f7656c4076a30653214f3cb514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cba9cd890a94c0f8e009058a026a729" is_pre_election: true
I20260619 02:43:15.247373 22145 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926e10f7656c4076a30653214f3cb514 in term 0.
I20260619 02:43:15.247550 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "926e10f7656c4076a30653214f3cb514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" is_pre_election: true
I20260619 02:43:15.247601 22218 leader_election.cc:302] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cba9cd890a94c0f8e009058a026a729, 926e10f7656c4076a30653214f3cb514; no voters:
I20260619 02:43:15.247651 22415 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926e10f7656c4076a30653214f3cb514 in term 0.
I20260619 02:43:15.247743 22475 raft_consensus.cc:2781] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:15.247813 22475 raft_consensus.cc:482] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:15.247839 22475 raft_consensus.cc:3037] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:15.248477 22475 raft_consensus.cc:504] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.248606 22475 leader_election.cc:288] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [CANDIDATE]: Term 1 election: Requested vote from peers a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065), 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897)
I20260619 02:43:15.248752 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "926e10f7656c4076a30653214f3cb514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
I20260619 02:43:15.248791 22145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "926e10f7656c4076a30653214f3cb514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cba9cd890a94c0f8e009058a026a729"
I20260619 02:43:15.248818 22415 raft_consensus.cc:3037] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:15.248860 22145 raft_consensus.cc:3037] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:15.249488 22415 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926e10f7656c4076a30653214f3cb514 in term 1.
I20260619 02:43:15.249670 22216 leader_election.cc:302] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 926e10f7656c4076a30653214f3cb514, a567e6ab6f1d4118bcc8d3b28270939e; no voters:
I20260619 02:43:15.249752 22475 raft_consensus.cc:2781] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:15.249855 22145 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926e10f7656c4076a30653214f3cb514 in term 1.
I20260619 02:43:15.249892 22475 raft_consensus.cc:686] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 LEADER]: Becoming Leader. State: Replica: 926e10f7656c4076a30653214f3cb514, State: Running, Role: LEADER
I20260619 02:43:15.249971 22475 consensus_queue.cc:237] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.250706 22003 catalog_manager.cc:5495] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 reported cstate change: term changed from 0 to 1, leader changed from <none> to 926e10f7656c4076a30653214f3cb514 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "926e10f7656c4076a30653214f3cb514" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } health_report { overall_health: UNKNOWN } } }
I20260619 02:43:15.261813 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.259545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.7.137.2:38953)
0619 02:43:15.259599 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:15.259601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:15.259609 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:15.259674 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:15.259675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:15.259690 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:15.259748 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:15.259854 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.260339 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.261088 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:15.261268 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:15.261323 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:15.261490 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:15.261578 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:15.261636 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:15.261677 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 02:43:15.261694 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:43:15.263628 22145 raft_consensus.cc:1261] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Refusing update from remote peer 926e10f7656c4076a30653214f3cb514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:15.263628 22415 raft_consensus.cc:1261] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Refusing update from remote peer 926e10f7656c4076a30653214f3cb514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:15.264019 22475 consensus_queue.cc:1035] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:15.264171 22478 consensus_queue.cc:1035] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:15.266495 22482 mvcc.cc:205] Tried to move back new op lower bound from 7298404332597092352 to 7298404332544393216. Current Snapshot: MvccSnapshot[applied={T|T < 7298404332597092352}]
I20260619 02:43:15.266932 22483 mvcc.cc:205] Tried to move back new op lower bound from 7298404332597092352 to 7298404332544393216. Current Snapshot: MvccSnapshot[applied={T|T < 7298404332597092352}]
I20260619 02:43:15.268405 22003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:15.268486 22003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:15.269268 22003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Sending DeleteTablet for 3 replicas of tablet 6626c9b76973439a9904c6ad3a135530
I20260619 02:43:15.269546 22263 tablet_service.cc:1514] Processing DeleteTablet for tablet 6626c9b76973439a9904c6ad3a135530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:15 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:49508
I20260619 02:43:15.269559 22126 tablet_service.cc:1514] Processing DeleteTablet for tablet 6626c9b76973439a9904c6ad3a135530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:15 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:57182
I20260619 02:43:15.269670 22200 tablet_replica.cc:330] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: stopping tablet replica
I20260619 02:43:15.269721 22200 raft_consensus.cc:2229] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:15.269761 22200 raft_consensus.cc:2258] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:15.270004 22395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6626c9b76973439a9904c6ad3a135530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:15 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:48192
I20260619 02:43:15.270045 22200 ts_tablet_manager.cc:1905] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:15.270198 22492 tablet_replica.cc:330] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: stopping tablet replica
I20260619 02:43:15.270223 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22041
I20260619 02:43:15.270711 22492 raft_consensus.cc:2229] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:15.270874 22492 raft_consensus.cc:2258] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:15.271353 22492 ts_tablet_manager.cc:1905] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:15.273401 22492 ts_tablet_manager.cc:1918] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:15.273522 22492 log.cc:1196] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/6626c9b76973439a9904c6ad3a135530
I20260619 02:43:15.273864 22492 ts_tablet_manager.cc:1939] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Deleting consensus metadata
I20260619 02:43:15.274336 21987 catalog_manager.cc:4841] TS a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065): tablet 6626c9b76973439a9904c6ad3a135530 (table test-table [id=ab52927938c14e6da6da8c742748a38c]) successfully deleted
W20260619 02:43:15.277022 21989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:41897 (error 108)
I20260619 02:43:15.277089 22491 tablet_replica.cc:330] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: stopping tablet replica
I20260619 02:43:15.277186 22491 raft_consensus.cc:2229] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:15.277197 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22197
I20260619 02:43:15.277281 22491 raft_consensus.cc:2258] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:15.278604 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.278222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.0.0.1:57192)
0619 02:43:15.278521 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.278570 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268}
W20260619 02:43:15.279641 21989 catalog_manager.cc:4584] TS 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6626c9b76973439a9904c6ad3a135530: Network error: Client connection negotiation failed: client connection to 127.7.137.1:41897: connect: Connection refused (error 111)
I20260619 02:43:15.283421 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.283293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:38953 (local address 127.0.0.1:49530)
0619 02:43:15.283351 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.283385 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.2:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:43:15.283490 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22332
I20260619 02:43:15.287958 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 21970
I20260619 02:43:15.293219 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--webserver_interface=127.7.137.62
--webserver_port=33553
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:15.373683 22494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.373929 22494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.373966 22494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.375108 22494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:15.375181 22494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.375204 22494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.375224 22494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:15.375254 22494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:15.376775 22494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=33553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.377044 22494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.377278 22494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.381948 22502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.382019 22500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.381989 22499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.382128 22494 server_base.cc:1034] running on GCE node
I20260619 02:43:15.382377 22494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.382668 22494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.383809 22494 hybrid_clock.cc:648] HybridClock initialized: now 1781836995383789 us; error 36 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.386349 22494 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:15.386818 22494 webserver.cc:460] Webserver started at http://127.7.137.62:33553/ using document root <none> and password file <none>
I20260619 02:43:15.387056 22494 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.387130 22494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.388432 22494 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.389178 22509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.389346 22494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:15.389460 22494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "d1f5fd55f1a84b8f806ea69bef0106aa"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "194ba10ac691c48e25ecbd0ea647a87b"
server_key_iv: "40103bb574190dc7c3e624b11ded7b2f"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.389789 22494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.402602 22494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.404856 22494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.405006 22494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.408856 22494 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41949
I20260619 02:43:15.409013 22561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41949 every 8 connection(s)
I20260619 02:43:15.409284 22494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:15.410835 22562 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:43:15.411686 22562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap starting.
I20260619 02:43:15.417953 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22494
I20260619 02:43:15.418256 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:41897
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=44535
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:15.420096 22562 log.cc:824] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:15.423331 22562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:15.423681 22562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa: Bootstrap complete.
I20260619 02:43:15.427637 22562 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:15.428031 22562 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.428220 22562 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:15.428308 22562 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:15.428365 22562 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:15.428422 22562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:43:15.429939 22562 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:15.430055 22562 leader_election.cc:302] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1f5fd55f1a84b8f806ea69bef0106aa; no voters:
I20260619 02:43:15.430208 22562 leader_election.cc:288] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:43:15.430282 22566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:43:15.430444 22566 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [term 3 LEADER]: Becoming Leader. State: Replica: d1f5fd55f1a84b8f806ea69bef0106aa, State: Running, Role: LEADER
I20260619 02:43:15.430478 22562 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:15.430531 22566 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } }
I20260619 02:43:15.430783 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:15.430855 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:15.431155 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f5fd55f1a84b8f806ea69bef0106aa. Latest consensus state: current_term: 3 leader_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f5fd55f1a84b8f806ea69bef0106aa" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41949 } } }
I20260619 02:43:15.431290 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f5fd55f1a84b8f806ea69bef0106aa [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:15.431236 22572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:15.432281 22572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:15.432387 22572 catalog_manager.cc:1259] Loaded cluster ID: f38b4f09743c4d67bafce2f0a9cd065c
I20260619 02:43:15.432423 22572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 02:43:15.432636 22572 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0f5cd269ad google::LogMessage::Fail() at ??:0
@ 0x7f0f5cd2988a google::LogMessage::SendToLog() at ??:0
@ 0x7f0f5cd264de google::LogMessage::Flush() at ??:0
@ 0x7f0f5cd27a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0f64c470f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0f64c508b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0f5ebf40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0f5ebe9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0f5fa4d6db start_thread at ??:0
@ 0x7f0f5b82461f clone at ??:0
W20260619 02:43:15.523566 22564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.523756 22564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.523779 22564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.525058 22564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.525099 22564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.525126 22564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:15.526527 22564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:41897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=44535
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.526794 22564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.527021 22564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.531852 22598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.531849 22599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.532106 22601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.532200 22564 server_base.cc:1034] running on GCE node
I20260619 02:43:15.532397 22564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.532629 22564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.533783 22564 hybrid_clock.cc:648] HybridClock initialized: now 1781836995533771 us; error 31 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.536283 22564 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:15.536767 22564 webserver.cc:460] Webserver started at http://127.7.137.1:44535/ using document root <none> and password file <none>
I20260619 02:43:15.537006 22564 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.537073 22564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.538370 22564 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.539119 22608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.539294 22564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.539376 22564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "4cba9cd890a94c0f8e009058a026a729"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "6fe329243b56f268eea3aee8680944a4"
server_key_iv: "a56d34ab603f5adadafc03c5b5f6c906"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.539711 22564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.554679 22564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.556924 22564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.557114 22564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.557367 22564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:15.557832 22615 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:43:15.559254 22564 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:43:15.559314 22564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:43:15.559346 22564 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:43:15.559986 22564 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:43:15.560030 22564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:15.560073 22615 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Bootstrap starting.
I20260619 02:43:15.562039 22615 log.cc:824] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:15.565197 22615 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:15.565501 22615 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Bootstrap complete.
I20260619 02:43:15.565737 22615 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260619 02:43:15.567245 22564 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:41897
I20260619 02:43:15.567591 22564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:15.567824 22615 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.568169 22615 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cba9cd890a94c0f8e009058a026a729, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.568312 22615 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.568560 22615 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260619 02:43:15.569012 22723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:41897 every 8 connection(s)
I20260619 02:43:15.573973 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22564
I20260619 02:43:15.574265 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:38953
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=44827
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:15.642122 22726 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:43:15.568223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:47543)
0619 02:43:15.568789 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.568828 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:15.568979 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:15.641892 (+ 72913us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":71,"threads_started":1}
W20260619 02:43:15.642329 22724 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:41949 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:41949: Client connection negotiation failed: client connection to 127.7.137.62:41949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:43:15.642716 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.642560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:52381)
0619 02:43:15.642628 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.642684 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:43:15.643102 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.643008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:49807)
0619 02:43:15.643059 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.643083 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:43:15.643265 22724 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:43:15.657049 22728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.657266 22728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.657305 22728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.658444 22728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.658506 22728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.658555 22728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:15.660027 22728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:38953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=44827
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.660280 22728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.660502 22728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.665373 22733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.665385 22734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.665418 22728 server_base.cc:1034] running on GCE node
W20260619 02:43:15.665390 22736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.665789 22728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.665980 22728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.667155 22728 hybrid_clock.cc:648] HybridClock initialized: now 1781836995667151 us; error 36 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.669618 22728 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:15.669986 22728 webserver.cc:460] Webserver started at http://127.7.137.2:44827/ using document root <none> and password file <none>
I20260619 02:43:15.670164 22728 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.670204 22728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.671483 22728 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.672176 22743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.672374 22728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:15.672464 22728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "926e10f7656c4076a30653214f3cb514"
format_stamp: "Formatted at 2026-06-19 02:43:13 on dist-test-slave-pdnb"
server_key: "cfaf30128b91ec943ee7dccb80d1fcf0"
server_key_iv: "7365932a1203912e3fe9e3df9e310f41"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.672845 22728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.685626 22728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.687872 22728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.688032 22728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.688258 22728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:15.688673 22750 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:43:15.690171 22728 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:43:15.690228 22728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:43:15.690253 22728 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:43:15.690899 22728 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:43:15.690944 22728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:15.690985 22750 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Bootstrap starting.
I20260619 02:43:15.698714 22728 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:38953
I20260619 02:43:15.698743 22857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:38953 every 8 connection(s)
I20260619 02:43:15.699254 22728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:15.699550 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22728
I20260619 02:43:15.699864 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:38065
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:41949
--webserver_port=35317
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:15.700675 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.700274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:33973)
0619 02:43:15.700465 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.700540 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
W20260619 02:43:15.700938 22858 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:41949 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:41949: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
I20260619 02:43:15.701287 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.701148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:42391)
0619 02:43:15.701213 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.701246 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 02:43:15.701608 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.701496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:55481)
0619 02:43:15.701545 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.701583 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:43:15.701727 22858 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:43:15.703776 22750 log.cc:824] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:15.706060 22750 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:15.706350 22750 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Bootstrap complete.
I20260619 02:43:15.707291 22750 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260619 02:43:15.708520 22750 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.708842 22750 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926e10f7656c4076a30653214f3cb514, State: Initialized, Role: FOLLOWER
I20260619 02:43:15.709021 22750 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:15.709270 22750 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260619 02:43:15.709610 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.709468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:59005)
0619 02:43:15.709533 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.709572 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:43:15.710013 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.709899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:34163)
0619 02:43:15.709951 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.709982 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:43:15.783545 22861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:15.783753 22861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:15.783776 22861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:15.785002 22861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:15.785046 22861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:15.785073 22861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:15.786696 22861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:38065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=35317
--tserver_master_addrs=127.7.137.62:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:15.786942 22861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:15.787187 22861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:15.792148 22869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.792150 22868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:15.792119 22871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:15.792326 22861 server_base.cc:1034] running on GCE node
I20260619 02:43:15.792608 22861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:15.792881 22861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:15.794044 22861 hybrid_clock.cc:648] HybridClock initialized: now 1781836995794021 us; error 41 us; skew 500 ppm
Jun 19 02:43:15 dist-test-slave-pdnb krb5kdc[21423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:15.796492 22861 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:15.796970 22861 webserver.cc:460] Webserver started at http://127.7.137.3:35317/ using document root <none> and password file <none>
I20260619 02:43:15.797196 22861 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:15.797240 22861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:15.798591 22861 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 02:43:15.799166 22878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.799347 22861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.799417 22861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "a567e6ab6f1d4118bcc8d3b28270939e"
format_stamp: "Formatted at 2026-06-19 02:43:14 on dist-test-slave-pdnb"
server_key: "28741d8b134988229df55253954105c3"
server_key_iv: "25a076e23279fa7db3e8c51adc7e4f25"
server_key_version: "encryptionkey@0"
I20260619 02:43:15.799723 22861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:15.818512 22861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:15.820489 22861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:15.820607 22861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:15.820865 22861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:15.821151 22861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:15.821182 22861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.821203 22861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:15.821216 22861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:15.827033 22861 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:38065
I20260619 02:43:15.827080 22991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:38065 every 8 connection(s)
I20260619 02:43:15.827526 22861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:15.828635 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.828206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:37783)
0619 02:43:15.828396 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.828474 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
W20260619 02:43:15.829093 22992 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:41949 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:41949: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
I20260619 02:43:15.829389 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.829296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:59901)
0619 02:43:15.829346 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.829372 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:43:15.829731 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:15.829639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:49735)
0619 02:43:15.829685 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:15.829714 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:43:15.829833 22992 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:43:15.835956 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 22861
I20260619 02:43:16.643858 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.643630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:60337)
0619 02:43:16.643762 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.643797 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260619 02:43:16.710668 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.710428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:49723)
0619 02:43:16.710585 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.710625 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
I20260619 02:43:16.830376 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:16.830143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:42505)
0619 02:43:16.830284 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:16.830322 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 02:43:17.364835 22999 raft_consensus.cc:482] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:17.364956 22999 raft_consensus.cc:504] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:17.365275 22999 leader_election.cc:288] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953), a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065)
Jun 19 02:43:17 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.3@KRBTEST.COM
Jun 19 02:43:17 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.2@KRBTEST.COM
I20260619 02:43:17.375734 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.365419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:38065 (local address 127.7.137.1:52977)
0619 02:43:17.365769 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.365779 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:17.365846 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:17.366481 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:17.366482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:17.366963 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:17.367021 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:17.367028 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.368106 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.368107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:17.368814 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:17.368816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.369398 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.369399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:17.369536 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:17.370136 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:17.370147 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:17.371784 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:17.374105 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:17.374112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:17.374329 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:17.374587 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:17.374588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:17.374679 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:17.375152 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:17.375158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:17.375288 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 02:43:17.375395 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:43:17.375617 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"thread_start_us":293,"threads_started":1}
I20260619 02:43:17.375788 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.365589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:52977 (local address 127.7.137.3:38065)
0619 02:43:17.365757 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:17.365768 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:17.365834 (+ 66us) server_negotiation.cc:407] Connection header received
0619 02:43:17.366311 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:17.366313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:17.366351 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:17.366495 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:17.367149 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.368002 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.368895 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.369292 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.369425 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:17.371904 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:17.371908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:17.371908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:17.371919 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:17.374015 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374396 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:17.374399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:17.374399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:17.374518 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374729 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:17.374729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:17.374729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:17.375070 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:17.375140 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:17.375464 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 02:43:17.375667 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":50,"threads_started":1}
I20260619 02:43:17.375859 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.365568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:41209 (local address 127.7.137.2:38953)
0619 02:43:17.365728 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:17.365739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:17.365758 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:43:17.365875 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:17.365877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:17.365916 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:17.366066 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:17.366747 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.367655 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.368945 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.369373 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.369512 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:17.372301 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:17.372305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:17.372305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:17.372317 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:17.374241 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374750 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:17.374752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:17.374753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:17.374825 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:17.375012 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:17.375013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:17.375013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:17.375288 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:17.375376 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:17.375616 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:43:17.375741 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260619 02:43:17.375825 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.365418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:38953 (local address 127.7.137.1:41209)
0619 02:43:17.365553 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.365571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:17.365622 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:17.366120 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:17.366123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:17.366428 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:17.366637 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:17.366643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.367940 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.367941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:17.368825 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:17.368827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:17.369478 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:17.369479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:17.369548 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:17.370303 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:17.370308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:17.372200 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:17.374318 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:17.374320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:17.374527 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:17.374884 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:17.374885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:17.374886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:17.374945 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:17.375363 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:17.375364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:17.375429 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 02:43:17.375505 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:43:17.375699 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":96,"thread_start_us":65,"threads_started":1}
I20260619 02:43:17.377051 22812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "4cba9cd890a94c0f8e009058a026a729" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "926e10f7656c4076a30653214f3cb514" is_pre_election: true
W20260619 02:43:17.377144 22609 leader_election.cc:341] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065): Not found: Tablet not found: 6626c9b76973439a9904c6ad3a135530
I20260619 02:43:17.377210 22812 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cba9cd890a94c0f8e009058a026a729 in term 1.
I20260619 02:43:17.377461 22612 leader_election.cc:302] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cba9cd890a94c0f8e009058a026a729, 926e10f7656c4076a30653214f3cb514; no voters: a567e6ab6f1d4118bcc8d3b28270939e
I20260619 02:43:17.377584 22999 raft_consensus.cc:2781] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:43:17.377643 22999 raft_consensus.cc:482] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:17.377671 22999 raft_consensus.cc:3037] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:43:17.378769 22999 raft_consensus.cc:504] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:17.378931 22999 leader_election.cc:288] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 election: Requested vote from peers 926e10f7656c4076a30653214f3cb514 (127.7.137.2:38953), a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065)
I20260619 02:43:17.379082 22812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6626c9b76973439a9904c6ad3a135530" candidate_uuid: "4cba9cd890a94c0f8e009058a026a729" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "926e10f7656c4076a30653214f3cb514"
W20260619 02:43:17.379179 22609 leader_election.cc:341] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065): Not found: Tablet not found: 6626c9b76973439a9904c6ad3a135530
I20260619 02:43:17.379189 22812 raft_consensus.cc:3037] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:43:17.380272 22812 raft_consensus.cc:2454] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cba9cd890a94c0f8e009058a026a729 in term 2.
I20260619 02:43:17.380420 22612 leader_election.cc:302] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cba9cd890a94c0f8e009058a026a729, 926e10f7656c4076a30653214f3cb514; no voters: a567e6ab6f1d4118bcc8d3b28270939e
I20260619 02:43:17.380538 22999 raft_consensus.cc:2781] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:43:17.380666 22999 raft_consensus.cc:686] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [term 2 LEADER]: Becoming Leader. State: Replica: 4cba9cd890a94c0f8e009058a026a729, State: Running, Role: LEADER
I20260619 02:43:17.380815 22999 consensus_queue.cc:237] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:17.381002 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.380855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:45207)
0619 02:43:17.380919 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.380956 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:43:17.711448 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.711180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:32933)
0619 02:43:17.711355 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.711400 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260619 02:43:17.831133 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:17.830838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:56015)
0619 02:43:17.831022 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:17.831079 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
W20260619 02:43:17.920323 22609 consensus_peers.cc:487] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 -> Peer a567e6ab6f1d4118bcc8d3b28270939e (127.7.137.3:38065): Couldn't send request to peer a567e6ab6f1d4118bcc8d3b28270939e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6626c9b76973439a9904c6ad3a135530. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:43:18.005221 22812 raft_consensus.cc:1261] T 6626c9b76973439a9904c6ad3a135530 P 926e10f7656c4076a30653214f3cb514 [term 2 FOLLOWER]: Refusing update from remote peer 4cba9cd890a94c0f8e009058a026a729: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 02:43:18.005636 23008 consensus_queue.cc:1035] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 02:43:18.006587 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.006444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:54025)
0619 02:43:18.006515 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.006554 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:43:18.007561 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.007465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:33433)
0619 02:43:18.007499 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.007527 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 02:43:18.007608 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.007412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:40047)
0619 02:43:18.007518 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.007553 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":89,"threads_started":1}
I20260619 02:43:18.409808 23015 ts_tablet_manager.cc:927] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Initiating tablet copy from peer 4cba9cd890a94c0f8e009058a026a729 (127.7.137.1:41897)
I20260619 02:43:18.410022 23015 tablet_copy_client.cc:316] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: tablet copy: Beginning tablet copy session from remote peer at address 127.7.137.1:41897
Jun 19 02:43:18 dist-test-slave-pdnb krb5kdc[21423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.1@KRBTEST.COM
I20260619 02:43:18.418356 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.410394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:56957 (local address 127.7.137.1:41897)
0619 02:43:18.410593 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:18.410599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:18.410611 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:43:18.410640 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:18.410643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:18.410658 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:18.410732 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:18.411253 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:18.411954 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:18.412853 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:18.413260 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:18.413354 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:18.415424 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:18.415438 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:18.415438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:18.415446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:18.417004 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:18.417412 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:18.417416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:18.417416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:18.417489 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:18.417690 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:18.417691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:18.417691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:18.417851 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:18.417939 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:18.418159 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 02:43:18.418241 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260619 02:43:18.418346 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.410297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:41897 (local address 127.7.137.3:56957)
0619 02:43:18.410457 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.410477 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:18.410578 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:18.410750 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:18.410754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:18.411086 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:18.411158 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:18.411160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:18.412051 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:18.412053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:18.412674 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:18.412676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:18.413356 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:18.413357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:18.413454 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:18.414003 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:18.414008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:18.415330 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:18.417090 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:18.417097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:18.417099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:18.417305 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:18.417563 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:18.417563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:18.417564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:18.417627 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:18.417929 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:18.417970 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:18.418027 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 02:43:18.418112 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:43:18.418241 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260619 02:43:18.418888 22698 tablet_copy_service.cc:140] P 4cba9cd890a94c0f8e009058a026a729: Received BeginTabletCopySession request for tablet 6626c9b76973439a9904c6ad3a135530 from peer a567e6ab6f1d4118bcc8d3b28270939e ({username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:56957)
I20260619 02:43:18.418957 22698 tablet_copy_service.cc:161] P 4cba9cd890a94c0f8e009058a026a729: Beginning new tablet copy session on tablet 6626c9b76973439a9904c6ad3a135530 from peer a567e6ab6f1d4118bcc8d3b28270939e at {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:56957: session id = a567e6ab6f1d4118bcc8d3b28270939e-6626c9b76973439a9904c6ad3a135530
I20260619 02:43:18.419466 22698 tablet_copy_source_session.cc:205] T 6626c9b76973439a9904c6ad3a135530 P 4cba9cd890a94c0f8e009058a026a729: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:43:18.420397 23015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6626c9b76973439a9904c6ad3a135530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:18.423117 23015 tablet_copy_client.cc:791] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: tablet copy: Starting download of 0 data blocks...
I20260619 02:43:18.423313 23015 tablet_copy_client.cc:655] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: tablet copy: Starting download of 1 WAL segments...
I20260619 02:43:18.424242 23015 tablet_copy_client.cc:531] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:43:18.425016 23015 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Bootstrap starting.
I20260619 02:43:18.426378 23015 log.cc:824] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:18.428889 23015 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:18.429131 23015 tablet_bootstrap.cc:492] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Bootstrap complete.
I20260619 02:43:18.429275 23015 ts_tablet_manager.cc:1397] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260619 02:43:18.430294 23015 raft_consensus.cc:348] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:18.430462 23015 raft_consensus.cc:729] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a567e6ab6f1d4118bcc8d3b28270939e, State: Initialized, Role: FOLLOWER
I20260619 02:43:18.430625 23015 consensus_queue.cc:260] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926e10f7656c4076a30653214f3cb514" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 38953 } } peers { permanent_uuid: "a567e6ab6f1d4118bcc8d3b28270939e" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38065 } } peers { permanent_uuid: "4cba9cd890a94c0f8e009058a026a729" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 41897 } }
I20260619 02:43:18.430934 23015 ts_tablet_manager.cc:1428] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:43:18.431288 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.431167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:59853)
0619 02:43:18.431222 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.431254 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:43:18.431285 22698 tablet_copy_service.cc:342] P 4cba9cd890a94c0f8e009058a026a729: Request end of tablet copy session a567e6ab6f1d4118bcc8d3b28270939e-6626c9b76973439a9904c6ad3a135530 received from {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:56957
I20260619 02:43:18.431341 22698 tablet_copy_service.cc:433] P 4cba9cd890a94c0f8e009058a026a729: ending tablet copy session a567e6ab6f1d4118bcc8d3b28270939e-6626c9b76973439a9904c6ad3a135530 on tablet 6626c9b76973439a9904c6ad3a135530 with peer a567e6ab6f1d4118bcc8d3b28270939e
I20260619 02:43:18.431841 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.431729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:47437)
0619 02:43:18.431780 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.431813 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 02:43:18.985414 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.985188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:54565)
0619 02:43:18.985307 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.985353 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260619 02:43:18.985361 22946 raft_consensus.cc:1203] T 6626c9b76973439a9904c6ad3a135530 P a567e6ab6f1d4118bcc8d3b28270939e [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 02:43:18.985977 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:18.985772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:59099)
0619 02:43:18.985877 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:18.985916 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260619 02:43:19.008189 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.007997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:38973)
0619 02:43:19.008099 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.008139 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
I20260619 02:43:19.986306 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.986033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:44549)
0619 02:43:19.986184 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.986223 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260619 02:43:19.986645 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:19.986424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:36345)
0619 02:43:19.986562 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:19.986594 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260619 02:43:20.009099 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.008702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:36505)
0619 02:43:20.009013 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.009050 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":254,"threads_started":1}
I20260619 02:43:20.987341 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.987078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:44319)
0619 02:43:20.987235 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.987293 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260619 02:43:20.987353 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.987137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:41587)
0619 02:43:20.987263 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.987309 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":44,"threads_started":1}
I20260619 02:43:20.987736 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:20.987650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:50231)
0619 02:43:20.987695 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:20.987718 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:43:21.009970 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.009667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:42151)
0619 02:43:21.009862 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.009899 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":122,"threads_started":1}
I20260619 02:43:21.987994 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.987788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:51451)
0619 02:43:21.987910 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.987948 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260619 02:43:21.988358 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:21.988157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:36903)
0619 02:43:21.988272 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:21.988305 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260619 02:43:22.010609 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.010423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:39895)
0619 02:43:22.010536 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.010561 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260619 02:43:22.988698 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.988471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:36611)
0619 02:43:22.988612 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.988650 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260619 02:43:22.989009 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:22.988817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:50185)
0619 02:43:22.988943 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:22.988973 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260619 02:43:23.011292 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.011077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:55017)
0619 02:43:23.011202 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.011242 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260619 02:43:23.989529 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.989249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:50457)
0619 02:43:23.989409 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.989461 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260619 02:43:23.989754 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:23.989532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:58311)
0619 02:43:23.989672 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:23.989704 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260619 02:43:24.011932 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.011727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:36459)
0619 02:43:24.011843 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.011881 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260619 02:43:24.990316 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.990012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:35377)
0619 02:43:24.990174 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.990227 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260619 02:43:24.990628 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:24.990332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:58279)
0619 02:43:24.990543 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:24.990588 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260619 02:43:25.012535 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.012322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:44507)
0619 02:43:25.012458 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.012488 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260619 02:43:25.991111 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.990819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:40507)
0619 02:43:25.990992 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.991043 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260619 02:43:25.991384 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:25.991112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:55957)
0619 02:43:25.991291 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:25.991335 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
I20260619 02:43:26.013226 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.012989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:48111)
0619 02:43:26.013137 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.013178 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260619 02:43:26.992033 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.991815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:35009)
0619 02:43:26.991948 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.991985 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260619 02:43:26.992097 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:26.991882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:37963)
0619 02:43:26.992019 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:26.992055 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260619 02:43:27.013947 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.013689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:56187)
0619 02:43:27.013850 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.013891 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260619 02:43:27.992831 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.992536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:55657)
0619 02:43:27.992686 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.992760 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260619 02:43:27.992884 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:27.992639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:38243)
0619 02:43:27.992807 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:27.992843 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260619 02:43:28.014709 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.014508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:39549)
0619 02:43:28.014627 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.014662 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260619 02:43:28.993587 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.993345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:33573)
0619 02:43:28.993510 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.993543 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
I20260619 02:43:28.993577 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:28.993345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:48289)
0619 02:43:28.993482 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:28.993526 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260619 02:43:29.015339 23059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:29.015117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:58247)
0619 02:43:29.015246 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:29.015287 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":52,"threads_started":1}
I20260619 02:43:29.994275 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:29.994058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.1:34593)
0619 02:43:29.994190 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:29.994227 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260619 02:43:29.994292 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:29.994058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.3:44331)
0619 02:43:29.994217 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:29.994246 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260619 02:43:30.015985 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:30.015783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41949 (local address 127.7.137.2:51871)
0619 02:43:30.015913 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:30.015937 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260619 02:43:30.837993 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22564
I20260619 02:43:30.842514 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22728
I20260619 02:43:30.846992 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 22861
2026-06-19T02:43:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18469 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:43:30 dist-test-slave-pdnb krb5kdc[23066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:43:30 dist-test-slave-pdnb krb5kdc[23066](info): set up 2 sockets
Jun 19 02:43:30 dist-test-slave-pdnb krb5kdc[23066](info): commencing operation
krb5kdc: starting...
W20260619 02:43:31.892814 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:43:31 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:43:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:43:32.009644 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:32.089377 23079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:32.089617 23079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:32.089654 23079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:32.090803 23079 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:43:32.090863 23079 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:43:32.090884 23079 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:43:32.090903 23079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:32.090931 23079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:32.090961 23079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:32.090984 23079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:32.091015 23079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:32.092468 23079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:32.092759 23079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:32.092978 23079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:32.097973 23087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.098145 23079 server_base.cc:1034] running on GCE node
W20260619 02:43:32.097973 23085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.098076 23084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.098492 23079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:32.098762 23079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:32.099895 23079 hybrid_clock.cc:648] HybridClock initialized: now 1781837012099882 us; error 30 us; skew 500 ppm
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:32.102375 23079 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:32.102855 23079 webserver.cc:460] Webserver started at http://127.7.137.62:33313/ using document root <none> and password file <none>
I20260619 02:43:32.103088 23079 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:32.103133 23079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:32.103276 23079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:32.104331 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "61f0e4f9174f45fa9410820578bfaa6e"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "4bcf3a1d205ace9726789828a874a58c"
server_key_iv: "6ce9c2548f807d430e5efbb63b228653"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.104689 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "61f0e4f9174f45fa9410820578bfaa6e"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "4bcf3a1d205ace9726789828a874a58c"
server_key_iv: "6ce9c2548f807d430e5efbb63b228653"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.106056 23079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:32.106853 23094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.107048 23079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:32.107138 23079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "61f0e4f9174f45fa9410820578bfaa6e"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "4bcf3a1d205ace9726789828a874a58c"
server_key_iv: "6ce9c2548f807d430e5efbb63b228653"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.107215 23079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:32.127002 23079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:32.128921 23079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:32.129067 23079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:32.132959 23079 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45327
I20260619 02:43:32.132993 23146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45327 every 8 connection(s)
I20260619 02:43:32.133342 23079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:32.134183 23147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.134225 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23079
I20260619 02:43:32.134323 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:43:32.134500 7716 external_mini_cluster.cc:1428] Setting key 61e510370a70e4bd0c52b202825e8fa6
I20260619 02:43:32.137092 23147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap starting.
I20260619 02:43:32.137718 23147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.137996 23147 log.cc:824] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.138660 23147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: No bootstrap required, opened a new log
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:32.140857 23147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:32.141021 23147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.141069 23147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.141214 23147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:32.141310 23147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:32.141342 23147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:32.141398 23147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.142024 23147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:32.142132 23147 leader_election.cc:302] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61f0e4f9174f45fa9410820578bfaa6e; no voters:
I20260619 02:43:32.142356 23147 leader_election.cc:288] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:43:32.142532 23153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:32.142630 23147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:32.142705 23153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 LEADER]: Becoming Leader. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Running, Role: LEADER
I20260619 02:43:32.142894 23153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:32.143389 23152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:43:32.143468 23152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:32.143585 23152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61f0e4f9174f45fa9410820578bfaa6e. Latest consensus state: current_term: 1 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:43:32.143630 23152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
W20260619 02:43:32.144483 23168 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:43:32.144548 23168 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:43:32.144645 23161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:32.144827 23161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:32.145463 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.135012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.7.137.62:45327)
0619 02:43:32.135195 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.135205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.135242 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:43:32.135700 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.135707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.135753 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.135896 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.136507 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.137475 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.138339 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.138775 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.138891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.141544 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.141547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.141548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:32.141557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.143679 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.144105 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.144107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.144108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.144213 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.144434 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.144435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.144435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.144758 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.144924 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.145025 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.145257 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260619 02:43:32.146265 23161 catalog_manager.cc:1347] Generated new cluster ID: f140f7ca36634a45832bcc847668bc5b
I20260619 02:43:32.146315 23161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:32.159888 23161 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:43:32.160372 23161 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:32.169945 23161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Generated new TSK 0
I20260619 02:43:32.170131 23161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:43:32.203449 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:32.305274 23173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:32.305509 23173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:32.305548 23173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:32.306696 23173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:32.306763 23173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:32.306810 23173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:32.308302 23173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:32.308559 23173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:32.308841 23173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:32.313939 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.313964 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.314095 23173 server_base.cc:1034] running on GCE node
W20260619 02:43:32.313939 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.314388 23173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:32.314637 23173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:32.315796 23173 hybrid_clock.cc:648] HybridClock initialized: now 1781837012315785 us; error 45 us; skew 500 ppm
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:32.318418 23173 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:32.318859 23173 webserver.cc:460] Webserver started at http://127.7.137.1:35797/ using document root <none> and password file <none>
I20260619 02:43:32.319083 23173 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:32.319129 23173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:32.319284 23173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:32.320336 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "1432e3760ec64923a684fee743d55fdf"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "20afe058636596d4282b11180364fb90"
server_key_iv: "9512c216312df8919b4eb610b0a9f2c4"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.320695 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "1432e3760ec64923a684fee743d55fdf"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "20afe058636596d4282b11180364fb90"
server_key_iv: "9512c216312df8919b4eb610b0a9f2c4"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.322036 23173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:32.322732 23188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.322912 23173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:32.322999 23173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "1432e3760ec64923a684fee743d55fdf"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "20afe058636596d4282b11180364fb90"
server_key_iv: "9512c216312df8919b4eb610b0a9f2c4"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.323067 23173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:32.331809 23173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:32.334021 23173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:32.334173 23173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:32.334398 23173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:32.334688 23173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:32.334723 23173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.334762 23173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:32.334789 23173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.340582 23173 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:44883
I20260619 02:43:32.340629 23301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:44883 every 8 connection(s)
I20260619 02:43:32.340968 23173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:32.348791 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23173
I20260619 02:43:32.348919 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:43:32.349089 7716 external_mini_cluster.cc:1428] Setting key 0a85ca72494fbcfe02013b32294ed1ba
I20260619 02:43:32.350513 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.341635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:35965 (local address 127.7.137.62:45327)
0619 02:43:32.341694 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.341696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.341906 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:43:32.342473 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.342475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.342486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.342548 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.343596 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.344157 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.345230 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.345446 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.345528 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.347545 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.347547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.347547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:32.347554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.349144 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.349549 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.349550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.349550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.349604 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.349893 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.349894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.349894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.350001 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.350055 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.350381 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.350449 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:43:32.350814 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.341604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:35965)
0619 02:43:32.341796 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.341830 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.342374 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.342669 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.342672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.343281 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:32.343476 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.343480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.344272 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.344274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.345139 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.345142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.345538 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.345539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.345655 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.346151 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:32.346159 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:32.347451 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:32.349220 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.349228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.349230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.349483 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.349670 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.349671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.349671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.349789 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.350065 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:32.350072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:32.350240 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.350353 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.350530 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260619 02:43:32.351327 23302 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:32.351439 23302 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:32.351637 23302 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:32.352136 23111 ts_manager.cc:194] Registered new tserver with Master: 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883)
I20260619 02:43:32.352806 23111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:35965
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:43:32.371541 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:32.463573 23307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:32.463860 23307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:32.463900 23307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:32.465636 23307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:32.465735 23307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:32.465814 23307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:32.467938 23307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:32.468307 23307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:32.468581 23307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:32.474504 23315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.474500 23312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.474696 23313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.478035 23307 server_base.cc:1034] running on GCE node
I20260619 02:43:32.478235 23307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:32.478482 23307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:32.479636 23307 hybrid_clock.cc:648] HybridClock initialized: now 1781837012479621 us; error 35 us; skew 500 ppm
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:32.481962 23307 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:32.482415 23307 webserver.cc:460] Webserver started at http://127.7.137.2:42169/ using document root <none> and password file <none>
I20260619 02:43:32.482646 23307 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:32.482710 23307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:32.482851 23307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:32.483911 23307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "f3ac516c9de84b1db3e8823252397280"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "7324b80eef4e238faffbc3d683cce6e9"
server_key_iv: "3d685e6b371a1c8f486310b634b1b372"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.484264 23307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "f3ac516c9de84b1db3e8823252397280"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "7324b80eef4e238faffbc3d683cce6e9"
server_key_iv: "3d685e6b371a1c8f486310b634b1b372"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.485625 23307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:43:32.486307 23322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.486469 23307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.486562 23307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "f3ac516c9de84b1db3e8823252397280"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "7324b80eef4e238faffbc3d683cce6e9"
server_key_iv: "3d685e6b371a1c8f486310b634b1b372"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.486637 23307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:32.522277 23307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:32.524091 23307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:32.524245 23307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:32.524473 23307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:32.524827 23307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:32.524883 23307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.524926 23307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:32.524955 23307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.530910 23307 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:44703
I20260619 02:43:32.530959 23435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:44703 every 8 connection(s)
I20260619 02:43:32.531291 23307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:43:32.536624 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23307
I20260619 02:43:32.536700 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:43:32.536901 7716 external_mini_cluster.cc:1428] Setting key 590e9224c56409a585d1e9fca9e6ccc3
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:32.540392 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.531996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:36573 (local address 127.7.137.62:45327)
0619 02:43:32.532057 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.532059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.532240 (+ 181us) server_negotiation.cc:407] Connection header received
0619 02:43:32.532707 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.532708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.532756 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.532845 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.533740 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.534230 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.535191 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.535364 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.535412 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.537528 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.537529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.537529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:32.537540 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.539110 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.539486 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.539488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.539488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.539551 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.539768 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.539768 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.539769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.539896 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.539955 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.540266 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.540330 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:43:32.540707 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.531973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:36573)
0619 02:43:32.532116 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.532157 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.532629 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.532903 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.532906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.533471 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:32.533654 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.533658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.534304 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.534305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.535121 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.535123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.535428 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.535429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.535524 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.535942 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:32.535950 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:32.537434 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:32.539188 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.539193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.539195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.539416 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.539620 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.539620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.539621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.539709 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.539964 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:32.539969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:32.540133 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.540245 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.540436 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260619 02:43:32.541304 23436 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:32.541394 23436 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:32.541579 23436 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:32.541980 23111 ts_manager.cc:194] Registered new tserver with Master: f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
I20260619 02:43:32.542600 23111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:36573
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:43:32.561054 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:32.640667 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:32.640926 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:32.640966 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:32.642154 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:32.642205 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:32.642246 23441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:32.644235 23441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:32.644493 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:32.644709 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:32.649518 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.649528 23446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:32.649528 23449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:32.649710 23441 server_base.cc:1034] running on GCE node
I20260619 02:43:32.649958 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:32.650188 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:32.651342 23441 hybrid_clock.cc:648] HybridClock initialized: now 1781837012651334 us; error 36 us; skew 500 ppm
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:32.654500 23441 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:32.655005 23441 webserver.cc:460] Webserver started at http://127.7.137.3:34341/ using document root <none> and password file <none>
I20260619 02:43:32.655242 23441 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:32.655308 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:32.655442 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:43:32.656450 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "829617fadb71476c8d5def4a4e0b7176"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "6d67031dda18d8df2f66d8020937b7f2"
server_key_iv: "357d5a007d7cbb48740c504036763887"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.656838 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "829617fadb71476c8d5def4a4e0b7176"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "6d67031dda18d8df2f66d8020937b7f2"
server_key_iv: "357d5a007d7cbb48740c504036763887"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.658097 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:32.658766 23456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.658921 23441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:43:32.659013 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "829617fadb71476c8d5def4a4e0b7176"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "6d67031dda18d8df2f66d8020937b7f2"
server_key_iv: "357d5a007d7cbb48740c504036763887"
server_key_version: "encryptionkey@0"
I20260619 02:43:32.659093 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:32.673800 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:32.676090 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:32.676247 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:32.676465 23441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:32.676836 23441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:32.676889 23441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.676929 23441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:32.676959 23441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:32.682786 23441 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45799
I20260619 02:43:32.682855 23569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45799 every 8 connection(s)
I20260619 02:43:32.683185 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:43:32.685847 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23441
I20260619 02:43:32.685937 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:43:32.686115 7716 external_mini_cluster.cc:1428] Setting key 474d2937f032f2f5054cf228231d9dd8
Jun 19 02:43:32 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837012, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:32.692380 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.683911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:50433 (local address 127.7.137.62:45327)
0619 02:43:32.683953 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.683954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.684141 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:43:32.684617 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.684618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.684629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.684704 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.685700 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.686461 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.687620 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.687820 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.687879 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.690059 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.690060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.690061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:32.690066 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.691304 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.691648 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.691649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.691649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.691710 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.691903 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.691903 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.691903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.691998 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.692046 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.692259 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.692317 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:43:32.692554 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.683887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:50433)
0619 02:43:32.684035 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.684069 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.684537 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.684831 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.684834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.685389 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:32.685580 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.685585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.686578 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.686579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.687506 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.687509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.687892 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.687893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.687988 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.688505 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:32.688513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:32.689980 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:32.691371 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.691377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.691378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.691588 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.691763 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:32.691763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:32.691763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:32.691854 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:32.692046 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:32.692050 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:32.692158 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.692238 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.692353 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260619 02:43:32.693018 23570 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:32.693125 23570 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:32.693300 23570 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:32.693631 23111 ts_manager.cc:194] Registered new tserver with Master: 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799)
I20260619 02:43:32.694002 23111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:50433
I20260619 02:43:32.697806 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:32.704564 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.699110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.7.137.62:45327)
0619 02:43:32.699185 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.699187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.699272 (+ 85us) server_negotiation.cc:407] Connection header received
0619 02:43:32.699311 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.699312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.699320 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.699376 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:32.699881 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.700351 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.701106 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.701290 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.701356 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.702475 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:32.702476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:32.702476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:32.702482 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:32.703667 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.703957 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.703958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.703958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.704003 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:32.704161 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:32.704162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:32.704162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:32.704291 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:32.704365 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.704445 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.704501 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:43:32.705946 23111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:32.707450 23111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:32.714104 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.710220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.7.137.1:44883)
0619 02:43:32.710381 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.710384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.710394 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:43:32.710435 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.710437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.710452 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.710727 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.710800 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.711992 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713167 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.713691 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713787 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.713859 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.713979 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.714018 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 02:43:32.714094 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.709993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:44883 (local address 127.0.0.1:35702)
0619 02:43:32.710117 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.710126 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.710198 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.710562 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.710563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.710565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.710644 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.710647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.712110 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.712111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.713097 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.713099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.713791 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.713877 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.713891 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.713959 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.714016 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260619 02:43:32.714263 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.709767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45799 (local address 127.0.0.1:56720)
0619 02:43:32.710077 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.710126 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.710198 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.710440 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.710443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.710446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.710644 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.710647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.711742 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.711743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.713013 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.713015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.713993 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.714063 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.714067 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.714115 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.714162 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260619 02:43:32.714264 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.709767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44703 (local address 127.0.0.1:32924)
0619 02:43:32.710289 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.710298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.710339 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.710702 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.710703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.710705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.710781 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.710783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.712071 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.712072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.713170 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.713172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.714058 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.714059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.714127 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.714130 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.714172 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.714204 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260619 02:43:32.714430 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.709889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.7.137.2:44703)
0619 02:43:32.710163 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.710168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.710399 (+ 231us) server_negotiation.cc:407] Connection header received
0619 02:43:32.710511 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.710514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.710531 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.710602 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.710852 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.711966 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713233 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.713920 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713982 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.714029 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.714302 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.714348 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260619 02:43:32.714510 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.709878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.7.137.3:45799)
0619 02:43:32.710061 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.710068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.710285 (+ 217us) server_negotiation.cc:407] Connection header received
0619 02:43:32.710333 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.710335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.710353 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.710434 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.710770 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.711641 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713097 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.713826 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.713898 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.713956 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.714383 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.714426 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260619 02:43:32.715241 23370 tablet_service.cc:1467] Processing CreateTablet for tablet d34791526e084a61b1076209f6c82e2b (DEFAULT_TABLE table=test-table [id=e538036d9087478a8d52f95f07d47353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.715229 23504 tablet_service.cc:1467] Processing CreateTablet for tablet d34791526e084a61b1076209f6c82e2b (DEFAULT_TABLE table=test-table [id=e538036d9087478a8d52f95f07d47353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.715397 23236 tablet_service.cc:1467] Processing CreateTablet for tablet d34791526e084a61b1076209f6c82e2b (DEFAULT_TABLE table=test-table [id=e538036d9087478a8d52f95f07d47353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:32.715606 23370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d34791526e084a61b1076209f6c82e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.715606 23504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d34791526e084a61b1076209f6c82e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.715685 23236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d34791526e084a61b1076209f6c82e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:32.718842 23588 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Bootstrap starting.
I20260619 02:43:32.718914 23587 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Bootstrap starting.
I20260619 02:43:32.719451 23587 tablet_bootstrap.cc:654] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.719465 23588 tablet_bootstrap.cc:654] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.719944 23587 log.cc:824] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.719978 23588 log.cc:824] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.720319 23589 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Bootstrap starting.
I20260619 02:43:32.720649 23587 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: No bootstrap required, opened a new log
I20260619 02:43:32.720671 23588 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: No bootstrap required, opened a new log
I20260619 02:43:32.720726 23587 ts_tablet_manager.cc:1397] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:43:32.720767 23588 ts_tablet_manager.cc:1397] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:32.722246 23587 raft_consensus.cc:348] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.722289 23588 raft_consensus.cc:348] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.722404 23587 raft_consensus.cc:374] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.722415 23588 raft_consensus.cc:374] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.722452 23587 raft_consensus.cc:729] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829617fadb71476c8d5def4a4e0b7176, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.722478 23588 raft_consensus.cc:729] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1432e3760ec64923a684fee743d55fdf, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.722575 23587 consensus_queue.cc:260] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.722609 23588 consensus_queue.cc:260] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.722884 23570 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
I20260619 02:43:32.722895 23588 ts_tablet_manager.cc:1428] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:32.722908 23302 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
I20260619 02:43:32.722882 23587 ts_tablet_manager.cc:1428] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:32.723564 23589 tablet_bootstrap.cc:654] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:32.724170 23589 log.cc:824] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:32.724849 23589 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: No bootstrap required, opened a new log
I20260619 02:43:32.724903 23589 ts_tablet_manager.cc:1397] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 02:43:32.726475 23589 raft_consensus.cc:348] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.726609 23589 raft_consensus.cc:374] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:32.726630 23589 raft_consensus.cc:729] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ac516c9de84b1db3e8823252397280, State: Initialized, Role: FOLLOWER
I20260619 02:43:32.726717 23589 consensus_queue.cc:260] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.726949 23589 ts_tablet_manager.cc:1428] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:32.726974 23436 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
W20260619 02:43:32.781832 23437 tablet.cc:2369] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:43:32.841670 23303 tablet.cc:2369] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:32.910693 23593 raft_consensus.cc:482] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:32.910846 23593 raft_consensus.cc:504] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.911228 23593 leader_election.cc:288] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703), 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799)
I20260619 02:43:32.915241 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.911476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:47153 (local address 127.7.137.3:45799)
0619 02:43:32.911528 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.911530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.911595 (+ 65us) server_negotiation.cc:407] Connection header received
0619 02:43:32.911661 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.911662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.911675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.911753 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.911963 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.912701 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914290 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.914874 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914946 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.914956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.915132 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.915169 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:43:32.915241 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.911460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:46665 (local address 127.7.137.2:44703)
0619 02:43:32.911510 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.911513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.911592 (+ 79us) server_negotiation.cc:407] Connection header received
0619 02:43:32.911661 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.911662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.911675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.911742 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:32.911963 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.912701 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914290 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.914870 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914946 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.914956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.915132 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.915168 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:43:32.915264 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.911386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45799 (local address 127.7.137.1:47153)
0619 02:43:32.911545 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.911553 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.911605 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.911775 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.911776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.911779 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.911891 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.911892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.912819 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.912819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.914202 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.914205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.914959 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.915050 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.915057 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.915118 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.915169 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":110,"thread_start_us":65,"threads_started":1}
I20260619 02:43:32.915267 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.911383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44703 (local address 127.7.137.1:46665)
0619 02:43:32.911514 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.911526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:32.911598 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:32.911775 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:32.911776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:32.911780 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:32.911886 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.911888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.912816 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.912817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.914204 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:32.914206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.914958 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.914959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:32.915050 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.915056 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:43:32.915118 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:43:32.915169 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":77,"thread_start_us":72,"threads_started":1}
I20260619 02:43:32.915616 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d34791526e084a61b1076209f6c82e2b" candidate_uuid: "1432e3760ec64923a684fee743d55fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829617fadb71476c8d5def4a4e0b7176" is_pre_election: true
I20260619 02:43:32.915782 23524 raft_consensus.cc:2454] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1432e3760ec64923a684fee743d55fdf in term 0.
I20260619 02:43:32.915807 23390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d34791526e084a61b1076209f6c82e2b" candidate_uuid: "1432e3760ec64923a684fee743d55fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ac516c9de84b1db3e8823252397280" is_pre_election: true
I20260619 02:43:32.916009 23390 raft_consensus.cc:2454] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1432e3760ec64923a684fee743d55fdf in term 0.
I20260619 02:43:32.916049 23189 leader_election.cc:302] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf, 829617fadb71476c8d5def4a4e0b7176; no voters:
I20260619 02:43:32.916205 23593 raft_consensus.cc:2781] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:32.916275 23593 raft_consensus.cc:482] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:32.916293 23593 raft_consensus.cc:3037] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.917011 23593 raft_consensus.cc:504] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.917148 23593 leader_election.cc:288] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 1 election: Requested vote from peers f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703), 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799)
I20260619 02:43:32.917315 23390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d34791526e084a61b1076209f6c82e2b" candidate_uuid: "1432e3760ec64923a684fee743d55fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ac516c9de84b1db3e8823252397280"
I20260619 02:43:32.917333 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d34791526e084a61b1076209f6c82e2b" candidate_uuid: "1432e3760ec64923a684fee743d55fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829617fadb71476c8d5def4a4e0b7176"
I20260619 02:43:32.917389 23390 raft_consensus.cc:3037] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.917438 23524 raft_consensus.cc:3037] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:32.918066 23390 raft_consensus.cc:2454] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1432e3760ec64923a684fee743d55fdf in term 1.
I20260619 02:43:32.918191 23524 raft_consensus.cc:2454] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1432e3760ec64923a684fee743d55fdf in term 1.
I20260619 02:43:32.918229 23192 leader_election.cc:302] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf, f3ac516c9de84b1db3e8823252397280; no voters:
I20260619 02:43:32.918327 23593 raft_consensus.cc:2781] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:32.918509 23593 raft_consensus.cc:686] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 LEADER]: Becoming Leader. State: Replica: 1432e3760ec64923a684fee743d55fdf, State: Running, Role: LEADER
I20260619 02:43:32.918617 23593 consensus_queue.cc:237] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:32.919359 23110 catalog_manager.cc:5495] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1432e3760ec64923a684fee743d55fdf (127.7.137.1). New cstate: current_term: 1 leader_uuid: "1432e3760ec64923a684fee743d55fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } health_report { overall_health: UNKNOWN } } }
W20260619 02:43:32.933704 23571 tablet.cc:2369] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:32.938459 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.936363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.7.137.1:44883)
0619 02:43:32.936419 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:32.936421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:32.936428 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:43:32.936461 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:32.936461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:32.936479 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:32.936537 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:32.936621 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.937122 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.937820 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:32.937994 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:32.938042 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:32.938151 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:32.938262 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:32.938318 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:32.938346 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:43:32.938362 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:43:32.940186 23524 raft_consensus.cc:1261] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Refusing update from remote peer 1432e3760ec64923a684fee743d55fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:32.940189 23390 raft_consensus.cc:1261] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Refusing update from remote peer 1432e3760ec64923a684fee743d55fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:32.940577 23598 consensus_queue.cc:1035] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:32.940711 23593 consensus_queue.cc:1035] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:32.943045 23603 mvcc.cc:205] Tried to move back new op lower bound from 7298404405000151040 to 7298404404915236864. Current Snapshot: MvccSnapshot[applied={T|T < 7298404405000151040}]
I20260619 02:43:32.943424 23602 mvcc.cc:205] Tried to move back new op lower bound from 7298404405000151040 to 7298404404915236864. Current Snapshot: MvccSnapshot[applied={T|T < 7298404405000151040}]
I20260619 02:43:32.945067 23110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:32.945166 23110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:32.945912 23110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Sending DeleteTablet for 3 replicas of tablet d34791526e084a61b1076209f6c82e2b
I20260619 02:43:32.946283 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23173
I20260619 02:43:32.946270 23504 tablet_service.cc:1514] Processing DeleteTablet for tablet d34791526e084a61b1076209f6c82e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:56720
I20260619 02:43:32.946273 23370 tablet_service.cc:1514] Processing DeleteTablet for tablet d34791526e084a61b1076209f6c82e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:32924
I20260619 02:43:32.947790 23612 tablet_replica.cc:330] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: stopping tablet replica
I20260619 02:43:32.947904 23612 raft_consensus.cc:2229] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:32.948012 23612 raft_consensus.cc:2258] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:32.948398 23612 ts_tablet_manager.cc:1905] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:32.950161 23612 ts_tablet_manager.cc:1918] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:32.950304 23612 log.cc:1196] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/d34791526e084a61b1076209f6c82e2b
I20260619 02:43:32.950591 23612 ts_tablet_manager.cc:1939] T d34791526e084a61b1076209f6c82e2b P f3ac516c9de84b1db3e8823252397280: Deleting consensus metadata
I20260619 02:43:32.950994 23098 catalog_manager.cc:4841] TS f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): tablet d34791526e084a61b1076209f6c82e2b (table test-table [id=e538036d9087478a8d52f95f07d47353]) successfully deleted
I20260619 02:43:32.952836 23611 tablet_replica.cc:330] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: stopping tablet replica
I20260619 02:43:32.952950 23611 raft_consensus.cc:2229] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:32.953068 23611 raft_consensus.cc:2258] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:32.953454 23611 ts_tablet_manager.cc:1905] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:43:32.953961 23098 connection.cc:535] client connection to 127.7.137.1:44883 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 02:43:32.954049 23098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:43:32.954259 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23307
I20260619 02:43:32.954849 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:32.954423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:44883 (local address 127.0.0.1:35716)
0619 02:43:32.954768 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:32.954809 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:44883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326}
I20260619 02:43:32.955116 23611 ts_tablet_manager.cc:1918] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:32.955196 23611 log.cc:1196] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/d34791526e084a61b1076209f6c82e2b
W20260619 02:43:32.955197 23098 catalog_manager.cc:4584] TS 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d34791526e084a61b1076209f6c82e2b: Network error: Client connection negotiation failed: client connection to 127.7.137.1:44883: connect: Connection refused (error 111)
I20260619 02:43:32.955444 23611 ts_tablet_manager.cc:1939] T d34791526e084a61b1076209f6c82e2b P 829617fadb71476c8d5def4a4e0b7176: Deleting consensus metadata
I20260619 02:43:32.955822 23095 catalog_manager.cc:4841] TS 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): tablet d34791526e084a61b1076209f6c82e2b (table test-table [id=e538036d9087478a8d52f95f07d47353]) successfully deleted
I20260619 02:43:32.959641 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23441
I20260619 02:43:32.963809 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23079
I20260619 02:43:52.969012 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--webserver_interface=127.7.137.62
--webserver_port=33313
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:43:53.051029 23614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:53.051229 23614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:53.051254 23614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:53.052393 23614 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:43:53.052434 23614 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:43:53.052448 23614 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:43:53.052459 23614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:43:53.052520 23614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:53.052538 23614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:53.052556 23614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:43:53.052575 23614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:43:53.054019 23614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=33313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:53.054271 23614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:53.054510 23614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:53.063490 23621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.063512 23614 server_base.cc:1034] running on GCE node
W20260619 02:43:53.063490 23623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.063485 23620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.064219 23614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:53.064529 23614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:53.065680 23614 hybrid_clock.cc:648] HybridClock initialized: now 1781837033065664 us; error 33 us; skew 500 ppm
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:53.072944 23614 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:43:53.073475 23614 webserver.cc:460] Webserver started at http://127.7.137.62:33313/ using document root <none> and password file <none>
I20260619 02:43:53.073740 23614 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:53.073832 23614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:53.075246 23614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:43:53.075978 23630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.076216 23614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:43:53.076301 23614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "61f0e4f9174f45fa9410820578bfaa6e"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "4bcf3a1d205ace9726789828a874a58c"
server_key_iv: "6ce9c2548f807d430e5efbb63b228653"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.076699 23614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:53.106307 23614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:53.108779 23614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:53.108922 23614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:53.112777 23614 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45327
I20260619 02:43:53.112797 23682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45327 every 8 connection(s)
I20260619 02:43:53.113267 23614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:43:53.114578 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23614
I20260619 02:43:53.114866 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:44883
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=35797
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:53.114878 23683 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:43:53.115585 23683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap starting.
I20260619 02:43:53.124001 23683 log.cc:824] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:53.126176 23683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:53.126492 23683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap complete.
I20260619 02:43:53.128800 23683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:53.129063 23683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.129251 23683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:53.129329 23683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:43:53.129386 23683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:43:53.129436 23683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:43:53.130407 23683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:53.130530 23683 leader_election.cc:302] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61f0e4f9174f45fa9410820578bfaa6e; no voters:
I20260619 02:43:53.130764 23683 leader_election.cc:288] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:43:53.130796 23687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:43:53.130910 23687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 LEADER]: Becoming Leader. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Running, Role: LEADER
I20260619 02:43:53.131026 23683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:43:53.131054 23687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:43:53.131423 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:43:53.131517 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:53.131420 23687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61f0e4f9174f45fa9410820578bfaa6e. Latest consensus state: current_term: 2 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:43:53.131716 23687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:43:53.132159 23695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:43:53.132951 23695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:43:53.133096 23695 catalog_manager.cc:1259] Loaded cluster ID: f140f7ca36634a45832bcc847668bc5b
I20260619 02:43:53.133147 23695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:43:53.133934 23695 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:43:53.134061 23695 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Loaded TSK: 0
I20260619 02:43:53.141170 23695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Generated new TSK 1
I20260619 02:43:53.141286 23695 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:43:53.210224 23685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:53.210456 23685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:53.210496 23685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:53.211625 23685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:53.211699 23685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:53.211746 23685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:43:53.213263 23685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:44883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=35797
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:53.213536 23685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:53.213768 23685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:53.218695 23710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.218737 23711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.218961 23685 server_base.cc:1034] running on GCE node
W20260619 02:43:53.219048 23713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.219257 23685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:53.219530 23685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:53.222858 23685 hybrid_clock.cc:648] HybridClock initialized: now 1781837033222828 us; error 42 us; skew 500 ppm
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:53.225916 23685 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:43:53.226477 23685 webserver.cc:460] Webserver started at http://127.7.137.1:35797/ using document root <none> and password file <none>
I20260619 02:43:53.226706 23685 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:53.226795 23685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:53.228292 23685 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:43:53.229102 23720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.229494 23685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:53.229596 23685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "1432e3760ec64923a684fee743d55fdf"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "20afe058636596d4282b11180364fb90"
server_key_iv: "9512c216312df8919b4eb610b0a9f2c4"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.229990 23685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:53.248158 23685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:53.250512 23685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:53.250679 23685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:53.250941 23685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:53.251387 23727 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:43:53.253350 23685 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:43:53.253423 23685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:43:53.253470 23685 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:43:53.254119 23685 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:43:53.254185 23685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:53.254257 23727 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Bootstrap starting.
I20260619 02:43:53.261399 23685 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:44883
I20260619 02:43:53.261464 23834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:44883 every 8 connection(s)
I20260619 02:43:53.261904 23685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:43:53.265357 23727 log.cc:824] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:53.267925 23727 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:43:53.268337 23727 tablet_bootstrap.cc:492] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Bootstrap complete.
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:53.269567 23727 ts_tablet_manager.cc:1397] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260619 02:43:53.270146 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23685
I20260619 02:43:53.270493 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:44703
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=42169
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:53.272737 23727 raft_consensus.cc:348] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:53.273066 23727 raft_consensus.cc:729] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1432e3760ec64923a684fee743d55fdf, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.273209 23727 consensus_queue.cc:260] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } } peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } }
I20260619 02:43:53.273528 23727 ts_tablet_manager.cc:1428] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:43:53.274082 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.262799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:36133)
0619 02:43:53.263141 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.263183 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.263711 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.264177 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.264182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.264471 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:53.264665 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.264672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.266122 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.266123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.267006 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.267009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.267650 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.267651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.267791 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.268402 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:53.268420 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:53.269911 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:53.272135 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.272142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.272144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.272469 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.272810 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.272811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.272811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.272947 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.273423 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:53.273429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:53.273569 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.273707 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.273884 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":95,"threads_started":1}
I20260619 02:43:53.274083 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.262863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:36133 (local address 127.7.137.62:45327)
0619 02:43:53.263049 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.263058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.263305 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:43:53.263938 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.263946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.263998 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.264140 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.264819 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.265983 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.267108 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.267513 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.267659 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.270071 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.270075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.270075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:53.270086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.272031 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.272572 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.272575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.272575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.272690 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.273027 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.273028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.273028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.273356 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.273441 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.273763 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.273884 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260619 02:43:53.275393 23835 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:53.275483 23835 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:53.275656 23835 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:53.276393 23647 ts_manager.cc:194] Registered new tserver with Master: 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883)
I20260619 02:43:53.277527 23647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:36133
I20260619 02:43:53.278698 23835 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
I20260619 02:43:53.281984 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.277836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.7.137.1:44883)
0619 02:43:53.277976 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.277981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.278319 (+ 338us) server_negotiation.cc:407] Connection header received
0619 02:43:53.278354 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.278356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.278521 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.278598 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:53.278909 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.279842 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.281063 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.281533 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.281646 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.281705 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.281878 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.281915 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260619 02:43:53.281985 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.277047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:44883 (local address 127.0.0.1:33254)
0619 02:43:53.278108 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.278161 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.278241 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.278658 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.278661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.278663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:53.278723 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.278725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.279933 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.279934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.280977 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.280980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.281651 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.281652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.281746 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.281758 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.281860 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.281915 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":87,"threads_started":1}
I20260619 02:43:53.282701 23769 tablet_service.cc:1514] Processing DeleteTablet for tablet d34791526e084a61b1076209f6c82e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:32 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:33254
I20260619 02:43:53.283213 23844 tablet_replica.cc:330] stopping tablet replica
I20260619 02:43:53.283332 23844 raft_consensus.cc:2229] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:53.283401 23844 raft_consensus.cc:2258] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:53.283701 23844 ts_tablet_manager.cc:1905] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:53.285941 23844 ts_tablet_manager.cc:1918] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:53.286127 23844 log.cc:1196] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal/wals/d34791526e084a61b1076209f6c82e2b
I20260619 02:43:53.286392 23844 ts_tablet_manager.cc:1939] T d34791526e084a61b1076209f6c82e2b P 1432e3760ec64923a684fee743d55fdf: Deleting consensus metadata
I20260619 02:43:53.286866 23634 catalog_manager.cc:4841] TS 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883): tablet d34791526e084a61b1076209f6c82e2b (table test-table [id=e538036d9087478a8d52f95f07d47353]) successfully deleted
W20260619 02:43:53.358454 23840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:53.358654 23840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:53.358676 23840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:53.359763 23840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:53.359797 23840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:53.359823 23840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:43:53.361212 23840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:44703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=42169
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:53.361392 23840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:53.361558 23840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:53.366681 23852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.366796 23840 server_base.cc:1034] running on GCE node
W20260619 02:43:53.366724 23849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.366715 23850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.367118 23840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:53.367355 23840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:53.368553 23840 hybrid_clock.cc:648] HybridClock initialized: now 1781837033368547 us; error 35 us; skew 500 ppm
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:53.371361 23840 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:43:53.371821 23840 webserver.cc:460] Webserver started at http://127.7.137.2:42169/ using document root <none> and password file <none>
I20260619 02:43:53.372066 23840 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:53.372148 23840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:53.373561 23840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:53.374161 23859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.374333 23840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:53.374436 23840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "f3ac516c9de84b1db3e8823252397280"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "7324b80eef4e238faffbc3d683cce6e9"
server_key_iv: "3d685e6b371a1c8f486310b634b1b372"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.374797 23840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:53.398886 23840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:53.401038 23840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:53.401198 23840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:53.401425 23840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:53.401731 23840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:53.401795 23840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.401836 23840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:53.401862 23840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.407898 23840 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:44703
I20260619 02:43:53.407958 23972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:44703 every 8 connection(s)
I20260619 02:43:53.408285 23840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:53.417322 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23840
I20260619 02:43:53.417479 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.409008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:33587 (local address 127.7.137.62:45327)
0619 02:43:53.409053 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.409055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.409258 (+ 203us) server_negotiation.cc:407] Connection header received
0619 02:43:53.409740 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.409742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.409754 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.409805 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.410712 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.411188 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.412166 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.412398 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.412457 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.414602 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.414603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.414603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:53.414610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.416025 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.416433 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.416434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.416434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.416485 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.416837 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.416838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.416838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.416941 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.416992 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.417278 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.417370 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:43:53.417632 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:45799
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=34341
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:43:53.417634 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.409006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:33587)
0619 02:43:53.409149 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.409189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.409640 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.409858 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.409862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.410418 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:53.410614 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.410619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.411280 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.411281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.412076 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.412078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.412499 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.412500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.412614 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.413120 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:53.413128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:53.414514 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:53.416100 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.416105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.416107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.416361 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.416574 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.416575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.416575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.416680 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.417003 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:53.417008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:53.417139 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.417229 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.417358 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260619 02:43:53.418488 23973 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:53.418578 23973 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:53.418774 23973 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:53.419210 23647 ts_manager.cc:194] Registered new tserver with Master: f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
I20260619 02:43:53.419741 23647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:33587
W20260619 02:43:53.498472 23976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:43:53.498740 23976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:43:53.498778 23976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:43:53.499913 23976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:43:53.499985 23976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:43:53.500028 23976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:43:53.501499 23976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:45799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=34341
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:43:53.501784 23976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:43:53.502013 23976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:43:53.507012 23981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:43:53.507014 23982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.507092 23976 server_base.cc:1034] running on GCE node
W20260619 02:43:53.507040 23984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:43:53.507485 23976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:43:53.507756 23976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:43:53.508925 23976 hybrid_clock.cc:648] HybridClock initialized: now 1781837033508893 us; error 50 us; skew 500 ppm
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:43:53.511369 23976 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:43:53.511830 23976 webserver.cc:460] Webserver started at http://127.7.137.3:34341/ using document root <none> and password file <none>
I20260619 02:43:53.512065 23976 fs_manager.cc:359] Metadata directory not provided
I20260619 02:43:53.512151 23976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:43:53.513641 23976 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:43:53.514262 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.514428 23976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:43:53.514535 23976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "829617fadb71476c8d5def4a4e0b7176"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "6d67031dda18d8df2f66d8020937b7f2"
server_key_iv: "357d5a007d7cbb48740c504036763887"
server_key_version: "encryptionkey@0"
I20260619 02:43:53.514878 23976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:43:53.536480 23976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:43:53.539080 23976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:43:53.539245 23976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:43:53.539491 23976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:43:53.539814 23976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:43:53.539876 23976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.539918 23976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:43:53.539947 23976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.545717 23976 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45799
I20260619 02:43:53.545821 24104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45799 every 8 connection(s)
I20260619 02:43:53.546156 23976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:43:53 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:43:53.553298 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 23976
I20260619 02:43:53.556947 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.547026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:42841 (local address 127.7.137.62:45327)
0619 02:43:53.547088 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.547090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.547335 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:43:53.547961 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.547962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.547972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.548029 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.549135 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.549618 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.550653 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.550848 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.550902 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.553548 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.553549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.553550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:53.553559 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.555384 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.555853 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.555854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.555855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.555918 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.556234 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.556235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.556235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.556359 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.556444 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.556771 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.556851 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:43:53.557147 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.547024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:42841)
0619 02:43:53.547193 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.547244 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.547834 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.548105 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.548108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.548835 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:43:53.549040 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.549044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.549723 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.549724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.550567 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.550570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.550925 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.550926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.551030 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.551547 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:43:53.551555 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:43:53.553377 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:43:53.555485 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.555490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.555493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.555726 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.555999 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:43:53.556000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:43:53.556000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:43:53.556119 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:43:53.556435 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:43:53.556442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:43:53.556586 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.556746 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.556913 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260619 02:43:53.557650 24105 heartbeater.cc:347] Connected to a master server at 127.7.137.62:45327
I20260619 02:43:53.557775 24105 heartbeater.cc:464] Registering TS with master...
I20260619 02:43:53.557976 24105 heartbeater.cc:511] Master 127.7.137.62:45327 requested a full tablet report, sending...
I20260619 02:43:53.558432 23647 ts_manager.cc:194] Registered new tserver with Master: 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799)
I20260619 02:43:53.558959 23647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:42841
I20260619 02:43:53.560596 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.553766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.7.137.62:45327)
0619 02:43:53.553900 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.553902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.553966 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:43:53.554064 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.554064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.554075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.554124 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.554841 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.555616 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.556406 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.556614 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.556692 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.558096 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.558097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.558097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:53.558106 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.559595 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.559933 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.559934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.559934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.559982 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.560146 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.560146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.560147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.560228 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.560381 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.560476 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.560535 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260619 02:43:53.561318 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:43:53.564065 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.561649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.7.137.62:45327)
0619 02:43:53.561710 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.561711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.561779 (+ 68us) server_negotiation.cc:407] Connection header received
0619 02:43:53.561821 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.561822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.561847 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.561904 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:53.562031 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.562484 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.563351 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.563532 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.563584 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.563820 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:53.563860 (+ 40us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 02:43:53.563991 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54250: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 02:43:53.564122 24109 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54250: token expired
W20260619 02:43:53.564213 23576 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.137.62:45327)
I20260619 02:43:53.570133 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.564427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.7.137.62:45327)
0619 02:43:53.564470 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.564472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.564481 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:43:53.564535 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.564536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.564545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.564592 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:43:53.565128 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.565571 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.566321 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.566517 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.566599 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.567744 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:43:53.567746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:43:53.567746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:43:53.567752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:43:53.568993 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.569348 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.569349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.569350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.569408 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:43:53.569615 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:43:53.569616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:43:53.569617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:43:53.569752 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:43:53.569858 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.569963 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.570039 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:43:53.571305 23647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:43:53.573032 23647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:43:53.575418 23769 tablet_service.cc:1467] Processing CreateTablet for tablet 20013b2a51fd4a03a1dbd5a666c5643d (DEFAULT_TABLE table=test-table [id=39e0f8c67df04cccb1028dde3ef9f4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:53.575620 23769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20013b2a51fd4a03a1dbd5a666c5643d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:53.577311 23727 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Bootstrap starting.
I20260619 02:43:53.577664 23727 tablet_bootstrap.cc:654] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:53.578265 23727 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: No bootstrap required, opened a new log
I20260619 02:43:53.578329 23727 ts_tablet_manager.cc:1397] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:43:53.578439 23727 raft_consensus.cc:348] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.578519 23727 raft_consensus.cc:374] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:53.578562 23727 raft_consensus.cc:729] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1432e3760ec64923a684fee743d55fdf, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.578608 23727 consensus_queue.cc:260] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.578706 23727 ts_tablet_manager.cc:1428] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 02:43:53.578941 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.574972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45799 (local address 127.0.0.1:45146)
0619 02:43:53.575076 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.575088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.575136 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.575465 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.575466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.575468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:53.575506 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.575507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.576661 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.576662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.577469 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.577473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.578680 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.578681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.578750 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.578754 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.578811 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.578856 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 02:43:53.578953 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.575107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.7.137.3:45799)
0619 02:43:53.575308 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.575314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.575325 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:43:53.575366 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.575368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.575385 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.575474 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:53.575562 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.576571 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.577547 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.578579 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.578674 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.578731 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.578834 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.578873 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":135,"threads_started":1}
I20260619 02:43:53.579352 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.574972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44703 (local address 127.0.0.1:54624)
0619 02:43:53.575318 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.575328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.575372 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.575621 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.575622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.575624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:53.575660 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.575661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.577011 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.577012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.578136 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.578138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.579123 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.579124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.579185 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.579189 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.579240 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.579277 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":20,"thread_start_us":108,"threads_started":1}
I20260619 02:43:53.579604 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.575105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54624 (local address 127.7.137.2:44703)
0619 02:43:53.575377 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.575381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.575393 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:43:53.575432 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.575434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.575449 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.575521 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:53.575757 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.576914 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.578248 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.578971 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.579036 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.579090 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.579467 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.579526 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":79,"threads_started":1}
I20260619 02:43:53.579798 24039 tablet_service.cc:1467] Processing CreateTablet for tablet 20013b2a51fd4a03a1dbd5a666c5643d (DEFAULT_TABLE table=test-table [id=39e0f8c67df04cccb1028dde3ef9f4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:53.580118 24039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20013b2a51fd4a03a1dbd5a666c5643d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:53.580229 23907 tablet_service.cc:1467] Processing CreateTablet for tablet 20013b2a51fd4a03a1dbd5a666c5643d (DEFAULT_TABLE table=test-table [id=39e0f8c67df04cccb1028dde3ef9f4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:43:53.580605 23907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20013b2a51fd4a03a1dbd5a666c5643d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:43:53.582892 24115 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Bootstrap starting.
I20260619 02:43:53.583159 24116 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Bootstrap starting.
I20260619 02:43:53.583846 24115 tablet_bootstrap.cc:654] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:53.583846 24116 tablet_bootstrap.cc:654] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Neither blocks nor log segments found. Creating new log.
I20260619 02:43:53.584491 24115 log.cc:824] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:53.584491 24116 log.cc:824] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Log is configured to *not* fsync() on all Append() calls
I20260619 02:43:53.585384 24116 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: No bootstrap required, opened a new log
I20260619 02:43:53.585402 24115 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: No bootstrap required, opened a new log
I20260619 02:43:53.585464 24116 ts_tablet_manager.cc:1397] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:43:53.585464 24115 ts_tablet_manager.cc:1397] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:43:53.587512 24115 raft_consensus.cc:348] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.587512 24116 raft_consensus.cc:348] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.587714 24116 raft_consensus.cc:374] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:53.587714 24115 raft_consensus.cc:374] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:43:53.587762 24115 raft_consensus.cc:729] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829617fadb71476c8d5def4a4e0b7176, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.587754 24116 raft_consensus.cc:729] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ac516c9de84b1db3e8823252397280, State: Initialized, Role: FOLLOWER
I20260619 02:43:53.587888 24115 consensus_queue.cc:260] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.587888 24116 consensus_queue.cc:260] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.588177 24115 ts_tablet_manager.cc:1428] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:43:53.588171 24116 ts_tablet_manager.cc:1428] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 02:43:53.588310 24105 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
I20260619 02:43:53.588328 23973 heartbeater.cc:503] Master 127.7.137.62:45327 was elected leader, sending a full tablet report...
W20260619 02:43:53.658878 23974 tablet.cc:2369] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:53.745069 24119 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:43:53.745193 24119 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.745591 24119 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703), 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883)
I20260619 02:43:53.748569 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.745855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44703 (local address 127.7.137.3:60687)
0619 02:43:53.745928 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.745942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.745993 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.746199 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.746200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.746202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:53.746288 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.746290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.746952 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.746953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.747707 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.747709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.748301 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.748302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.748384 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.748387 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.748458 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.748497 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:43:53.748569 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.745936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:60687 (local address 127.7.137.2:44703)
0619 02:43:53.745997 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.745999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.746050 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:43:53.746083 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.746084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.746098 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.746184 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:53.746368 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.746857 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.747813 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.748191 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.748296 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.748307 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.748464 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.748503 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:43:53.748950 24121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.745989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:44883 (local address 127.7.137.3:51407)
0619 02:43:53.746201 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.746210 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:43:53.746291 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:43:53.746452 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:43:53.746452 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:43:53.746454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:43:53.746529 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.746531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.747168 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.747169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.747918 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:43:53.747919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.748485 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.748486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:43:53.748557 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.748560 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:43:53.748816 (+ 256us) client_negotiation.cc:240] Negotiation successful
0619 02:43:53.748858 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":124,"threads_started":1}
I20260619 02:43:53.748939 23927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20013b2a51fd4a03a1dbd5a666c5643d" candidate_uuid: "829617fadb71476c8d5def4a4e0b7176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ac516c9de84b1db3e8823252397280" is_pre_election: true
I20260619 02:43:53.749111 23927 raft_consensus.cc:2454] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 829617fadb71476c8d5def4a4e0b7176 in term 0.
I20260619 02:43:53.749341 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.746083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:51407 (local address 127.7.137.1:44883)
0619 02:43:53.746127 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.746129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.746281 (+ 152us) server_negotiation.cc:407] Connection header received
0619 02:43:53.746347 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.746349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.746361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.746445 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:43:53.746604 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.747079 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.748017 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.748385 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.748481 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.748490 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.749235 (+ 745us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.749293 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:43:53.749347 23995 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280; no voters:
I20260619 02:43:53.749557 24119 raft_consensus.cc:2781] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:43:53.749621 24119 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:43:53.749639 24119 raft_consensus.cc:3037] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:53.749601 23789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20013b2a51fd4a03a1dbd5a666c5643d" candidate_uuid: "829617fadb71476c8d5def4a4e0b7176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1432e3760ec64923a684fee743d55fdf" is_pre_election: true
I20260619 02:43:53.749804 23789 raft_consensus.cc:2454] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 829617fadb71476c8d5def4a4e0b7176 in term 0.
I20260619 02:43:53.750427 24119 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.750562 24119 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [CANDIDATE]: Term 1 election: Requested vote from peers f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703), 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883)
I20260619 02:43:53.750744 23789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20013b2a51fd4a03a1dbd5a666c5643d" candidate_uuid: "829617fadb71476c8d5def4a4e0b7176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1432e3760ec64923a684fee743d55fdf"
I20260619 02:43:53.750756 23927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20013b2a51fd4a03a1dbd5a666c5643d" candidate_uuid: "829617fadb71476c8d5def4a4e0b7176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ac516c9de84b1db3e8823252397280"
I20260619 02:43:53.750842 23789 raft_consensus.cc:3037] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:53.750846 23927 raft_consensus.cc:3037] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:43:53.751609 23927 raft_consensus.cc:2454] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 829617fadb71476c8d5def4a4e0b7176 in term 1.
I20260619 02:43:53.751778 23995 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280; no voters:
I20260619 02:43:53.752040 23789 raft_consensus.cc:2454] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 829617fadb71476c8d5def4a4e0b7176 in term 1.
I20260619 02:43:53.752151 24119 raft_consensus.cc:2781] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:43:53.752338 24119 raft_consensus.cc:686] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 1 LEADER]: Becoming Leader. State: Replica: 829617fadb71476c8d5def4a4e0b7176, State: Running, Role: LEADER
I20260619 02:43:53.752451 24119 consensus_queue.cc:237] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:43:53.753396 23647 catalog_manager.cc:5495] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 reported cstate change: term changed from 0 to 1, leader changed from <none> to 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "829617fadb71476c8d5def4a4e0b7176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } health_report { overall_health: UNKNOWN } } }
I20260619 02:43:53.759573 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.757350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.7.137.3:45799)
0619 02:43:53.757415 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:43:53.757417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:43:53.757425 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:43:53.757492 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:43:53.757492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:43:53.757508 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:43:53.757567 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:43:53.757688 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.758126 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.758876 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:43:53.759059 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:43:53.759121 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:43:53.759242 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:43:53.759344 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:43:53.759405 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:43:53.759437 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 02:43:53.759452 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:43:53.761462 23927 raft_consensus.cc:1261] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Refusing update from remote peer 829617fadb71476c8d5def4a4e0b7176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:53.761467 23789 raft_consensus.cc:1261] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Refusing update from remote peer 829617fadb71476c8d5def4a4e0b7176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:43:53.761862 24119 consensus_queue.cc:1035] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:43:53.762017 24122 consensus_queue.cc:1035] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:43:53.763018 23836 tablet.cc:2369] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:43:53.764165 24125 mvcc.cc:205] Tried to move back new op lower bound from 7298404490284118016 to 7298404490250809344. Current Snapshot: MvccSnapshot[applied={T|T < 7298404490284118016}]
I20260619 02:43:53.764536 24127 mvcc.cc:205] Tried to move back new op lower bound from 7298404490284118016 to 7298404490250809344. Current Snapshot: MvccSnapshot[applied={T|T < 7298404490284118016}]
I20260619 02:43:53.764683 24126 mvcc.cc:205] Tried to move back new op lower bound from 7298404490284118016 to 7298404490250809344. Current Snapshot: MvccSnapshot[applied={T|T < 7298404490284118016}]
I20260619 02:43:53.766891 23646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:53.767027 23646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:43:53.767794 23646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Sending DeleteTablet for 3 replicas of tablet 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:43:53.768069 24039 tablet_service.cc:1514] Processing DeleteTablet for tablet 20013b2a51fd4a03a1dbd5a666c5643d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:53 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:45146
I20260619 02:43:53.768154 23907 tablet_service.cc:1514] Processing DeleteTablet for tablet 20013b2a51fd4a03a1dbd5a666c5643d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:43:53 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:54624
I20260619 02:43:53.768290 24135 tablet_replica.cc:330] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: stopping tablet replica
I20260619 02:43:53.768362 24136 tablet_replica.cc:330] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: stopping tablet replica
I20260619 02:43:53.768383 24135 raft_consensus.cc:2229] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:43:53.768437 24136 raft_consensus.cc:2229] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:43:53.768476 24135 raft_consensus.cc:2258] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:53.768523 24136 raft_consensus.cc:2258] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:43:53.768595 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23685
I20260619 02:43:53.769495 24136 ts_tablet_manager.cc:1905] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:53.771457 24136 ts_tablet_manager.cc:1918] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:53.771534 24136 log.cc:1196] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:43:53.771806 24136 ts_tablet_manager.cc:1939] T 20013b2a51fd4a03a1dbd5a666c5643d P f3ac516c9de84b1db3e8823252397280: Deleting consensus metadata
I20260619 02:43:53.772146 24135 ts_tablet_manager.cc:1905] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:43:53.772326 23634 catalog_manager.cc:4841] TS f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): tablet 20013b2a51fd4a03a1dbd5a666c5643d (table test-table [id=39e0f8c67df04cccb1028dde3ef9f4a8]) successfully deleted
I20260619 02:43:53.774394 24135 ts_tablet_manager.cc:1918] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:43:53.774473 24135 log.cc:1196] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:43:53.774775 24135 ts_tablet_manager.cc:1939] T 20013b2a51fd4a03a1dbd5a666c5643d P 829617fadb71476c8d5def4a4e0b7176: Deleting consensus metadata
I20260619 02:43:53.775187 23631 catalog_manager.cc:4841] TS 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): tablet 20013b2a51fd4a03a1dbd5a666c5643d (table test-table [id=39e0f8c67df04cccb1028dde3ef9f4a8]) successfully deleted
W20260619 02:43:53.776010 23634 connection.cc:535] client connection to 127.7.137.1:44883 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 02:43:53.776120 23634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:43:53.776468 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23840
I20260619 02:43:53.776618 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:43:53.776471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:44883 (local address 127.0.0.1:33264)
0619 02:43:53.776542 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:43:53.776581 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:44883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 02:43:53.778128 23634 catalog_manager.cc:4584] TS 1432e3760ec64923a684fee743d55fdf (127.7.137.1:44883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20013b2a51fd4a03a1dbd5a666c5643d: Network error: Client connection negotiation failed: client connection to 127.7.137.1:44883: connect: Connection refused (error 111)
I20260619 02:43:53.781690 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23976
I20260619 02:43:53.786010 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 23614
I20260619 02:44:13.791566 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--webserver_interface=127.7.137.62
--webserver_port=33313
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:45327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:13.872303 24138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:13.872509 24138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:13.872532 24138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:13.873624 24138 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:44:13.873657 24138 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:44:13.873670 24138 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:44:13.873682 24138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:13.873694 24138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:13.873708 24138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:13.873724 24138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:13.873737 24138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:13.875108 24138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:45327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:45327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=33313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:13.875286 24138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:13.875444 24138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:13.880475 24144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:13.880529 24138 server_base.cc:1034] running on GCE node
W20260619 02:44:13.880530 24143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:13.880491 24146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:13.880968 24138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:13.881225 24138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:13.882392 24138 hybrid_clock.cc:648] HybridClock initialized: now 1781837053882387 us; error 35 us; skew 500 ppm
Jun 19 02:44:13 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837053, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:13.884908 24138 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:13.885329 24138 webserver.cc:460] Webserver started at http://127.7.137.62:33313/ using document root <none> and password file <none>
I20260619 02:44:13.885550 24138 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:13.885617 24138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:13.887012 24138 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:13.887689 24153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:13.887872 24138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:13.887969 24138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "61f0e4f9174f45fa9410820578bfaa6e"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "4bcf3a1d205ace9726789828a874a58c"
server_key_iv: "6ce9c2548f807d430e5efbb63b228653"
server_key_version: "encryptionkey@0"
I20260619 02:44:13.888324 24138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:13.900700 24138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:13.902633 24138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:13.902782 24138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:13.906610 24138 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:45327
I20260619 02:44:13.906675 24205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:45327 every 8 connection(s)
I20260619 02:44:13.907035 24138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:13.908473 24206 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:44:13.909155 24206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap starting.
I20260619 02:44:13.916317 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24138
I20260619 02:44:13.916599 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:44883
--local_ip_for_outbound_sockets=127.7.137.1
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=35797
--webserver_interface=127.7.137.1
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:13.917510 24206 log.cc:824] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:13.919708 24206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:44:13.920025 24206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Bootstrap complete.
I20260619 02:44:13.922293 24206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:44:13.922510 24206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Initialized, Role: FOLLOWER
I20260619 02:44:13.922676 24206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:44:13.922736 24206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:13.922796 24206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:13.922842 24206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:44:13.923832 24206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:44:13.923918 24206 leader_election.cc:302] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61f0e4f9174f45fa9410820578bfaa6e; no voters:
I20260619 02:44:13.924144 24206 leader_election.cc:288] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:44:13.924178 24210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:44:13.924309 24210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [term 3 LEADER]: Becoming Leader. State: Replica: 61f0e4f9174f45fa9410820578bfaa6e, State: Running, Role: LEADER
I20260619 02:44:13.924412 24206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:13.924466 24210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } }
I20260619 02:44:13.924837 24210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61f0e4f9174f45fa9410820578bfaa6e. Latest consensus state: current_term: 3 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:44:13.925017 24212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "61f0e4f9174f45fa9410820578bfaa6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f0e4f9174f45fa9410820578bfaa6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 45327 } } }
I20260619 02:44:13.925102 24210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:13.925164 24212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:13.925590 24219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:13.926412 24219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:13.926555 24219 catalog_manager.cc:1259] Loaded cluster ID: f140f7ca36634a45832bcc847668bc5b
I20260619 02:44:13.926614 24219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:13.927429 24219 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:13.927631 24219 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Loaded TSK: 0
I20260619 02:44:13.927688 24219 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61f0e4f9174f45fa9410820578bfaa6e: Loaded TSK: 1
F20260619 02:44:13.927906 24219 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f489192c9ad google::LogMessage::Fail() at ??:0
@ 0x7f489192f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f489192c4de google::LogMessage::Flush() at ??:0
@ 0x7f489192da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4894099104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4894096d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f48998861cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f489988648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4899886594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f489984cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4899856ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f48937fa0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f48937ef954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f48946536db start_thread at ??:0
@ 0x7f489042a61f clone at ??:0
W20260619 02:44:14.036386 24208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:14.036685 24208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:14.036746 24208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:14.038699 24208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:14.038770 24208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:14.038818 24208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:14.041281 24208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:44883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=35797
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:14.041566 24208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:14.041867 24208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:14.047417 24247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:14.047493 24248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:14.047552 24208 server_base.cc:1034] running on GCE node
W20260619 02:44:14.047679 24250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:14.047880 24208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:14.048132 24208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:14.049346 24208 hybrid_clock.cc:648] HybridClock initialized: now 1781837054049322 us; error 52 us; skew 500 ppm
Jun 19 02:44:14 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:14.051926 24208 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:44:14.052349 24208 webserver.cc:460] Webserver started at http://127.7.137.1:35797/ using document root <none> and password file <none>
I20260619 02:44:14.052600 24208 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:14.052670 24208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:14.054103 24208 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:44:14.054852 24257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.055058 24208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:44:14.055168 24208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "1432e3760ec64923a684fee743d55fdf"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "20afe058636596d4282b11180364fb90"
server_key_iv: "9512c216312df8919b4eb610b0a9f2c4"
server_key_version: "encryptionkey@0"
I20260619 02:44:14.055522 24208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:14.064280 24208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:14.066509 24208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:14.066664 24208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:14.066943 24208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:14.067433 24264 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:44:14.069568 24208 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:44:14.069638 24208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 02:44:14.069684 24208 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:44:14.070304 24208 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:44:14.070359 24208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 02:44:14.070441 24264 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Bootstrap starting.
I20260619 02:44:14.077680 24208 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:44883
I20260619 02:44:14.077834 24371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:44883 every 8 connection(s)
I20260619 02:44:14.078122 24208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:44:14.081933 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24208
I20260619 02:44:14.082240 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:44703
--local_ip_for_outbound_sockets=127.7.137.2
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=42169
--webserver_interface=127.7.137.2
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:44:14.084098 24264 log.cc:824] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:14.087881 24264 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:44:14.088186 24264 tablet_bootstrap.cc:492] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Bootstrap complete.
I20260619 02:44:14.089390 24264 ts_tablet_manager.cc:1397] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.011s
I20260619 02:44:14.090835 24264 raft_consensus.cc:348] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:14.091166 24264 raft_consensus.cc:729] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1432e3760ec64923a684fee743d55fdf, State: Initialized, Role: FOLLOWER
I20260619 02:44:14.091416 24264 consensus_queue.cc:260] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:14.091898 24264 ts_tablet_manager.cc:1428] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260619 02:44:14.156836 24374 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:44:14.078885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:55817)
0619 02:44:14.079059 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.079102 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:14.079554 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:14.156612 (+ 77058us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
W20260619 02:44:14.157122 24372 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45327 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:45327: Client connection negotiation failed: client connection to 127.7.137.62:45327: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:44:14.157493 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.157371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:38065)
0619 02:44:14.157426 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.157468 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 02:44:14.157832 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.157721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:51293)
0619 02:44:14.157787 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.157814 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 02:44:14.157977 24372 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:44:14.169137 24375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:14.169355 24375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:14.169394 24375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:14.170554 24375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:14.170619 24375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:14.170666 24375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:14.172092 24375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:44703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=42169
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:14.172343 24375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:14.172570 24375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:14.177469 24383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:14.177460 24382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:14.177460 24385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:14.177683 24375 server_base.cc:1034] running on GCE node
I20260619 02:44:14.177923 24375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:14.178251 24375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:14.179392 24375 hybrid_clock.cc:648] HybridClock initialized: now 1781837054179356 us; error 49 us; skew 500 ppm
Jun 19 02:44:14 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:14.181988 24375 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:44:14.182359 24375 webserver.cc:460] Webserver started at http://127.7.137.2:42169/ using document root <none> and password file <none>
I20260619 02:44:14.182557 24375 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:14.182597 24375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:14.183930 24375 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:14.184512 24392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.184705 24375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:14.184827 24375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "f3ac516c9de84b1db3e8823252397280"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "7324b80eef4e238faffbc3d683cce6e9"
server_key_iv: "3d685e6b371a1c8f486310b634b1b372"
server_key_version: "encryptionkey@0"
I20260619 02:44:14.185160 24375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:14.207180 24375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:14.209070 24375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:14.209226 24375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:14.209463 24375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:14.209776 24375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:14.209808 24375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.209851 24375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:14.209878 24375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.215377 24375 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:44703
I20260619 02:44:14.215479 24505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:44703 every 8 connection(s)
I20260619 02:44:14.215803 24375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:14.216909 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.216475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:54399)
0619 02:44:14.216630 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.216698 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
W20260619 02:44:14.217307 24506 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45327 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:45327: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
I20260619 02:44:14.217597 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.217510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:39555)
0619 02:44:14.217554 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.217580 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:44:14.217928 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.217855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:50849)
0619 02:44:14.217895 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.217915 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:44:14.218056 24506 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:44:14.218601 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24375
I20260619 02:44:14.218827 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:45799
--local_ip_for_outbound_sockets=127.7.137.3
--tserver_master_addrs=127.7.137.62:45327
--webserver_port=34341
--webserver_interface=127.7.137.3
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:14.300130 24509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:14.300328 24509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:14.300351 24509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:14.301455 24509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:14.301489 24509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:14.301514 24509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:14.302922 24509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:45799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=34341
--tserver_master_addrs=127.7.137.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:14.303133 24509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:14.303315 24509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:14.308251 24515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:14.308292 24514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:14.308339 24509 server_base.cc:1034] running on GCE node
W20260619 02:44:14.308262 24517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:14.308668 24509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:14.308928 24509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:14.310094 24509 hybrid_clock.cc:648] HybridClock initialized: now 1781837054310088 us; error 42 us; skew 500 ppm
Jun 19 02:44:14 dist-test-slave-pdnb krb5kdc[23066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:14.312464 24509 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:44:14.312932 24509 webserver.cc:460] Webserver started at http://127.7.137.3:34341/ using document root <none> and password file <none>
I20260619 02:44:14.313171 24509 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:14.313236 24509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:14.314543 24509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:14.315145 24524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.315318 24509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:14.315420 24509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "829617fadb71476c8d5def4a4e0b7176"
format_stamp: "Formatted at 2026-06-19 02:43:32 on dist-test-slave-pdnb"
server_key: "6d67031dda18d8df2f66d8020937b7f2"
server_key_iv: "357d5a007d7cbb48740c504036763887"
server_key_version: "encryptionkey@0"
I20260619 02:44:14.315773 24509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:14.335216 24509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:14.337100 24509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:14.337261 24509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:14.337481 24509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:14.337795 24509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:14.337845 24509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.337886 24509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:14.337914 24509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:14.343339 24509 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:45799
I20260619 02:44:14.343439 24637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:45799 every 8 connection(s)
I20260619 02:44:14.343746 24509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:14.344760 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.344353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:48001)
0619 02:44:14.344497 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.344592 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260619 02:44:14.345120 24638 heartbeater.cc:644] Failed to heartbeat to 127.7.137.62:45327 (0 consecutive failures): Network error: Failed to ping master at 127.7.137.62:45327: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
I20260619 02:44:14.345371 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.345285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:52421)
0619 02:44:14.345327 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.345353 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:44:14.345613 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:14.345541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:45113)
0619 02:44:14.345578 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:14.345599 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:44:14.345716 24638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:44:14.353677 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24509
I20260619 02:44:15.158594 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.158379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:35509)
0619 02:44:15.158497 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:15.158535 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260619 02:44:15.218531 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.218337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:46667)
0619 02:44:15.218446 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:15.218483 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260619 02:44:15.346213 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.346015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:41611)
0619 02:44:15.346129 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:15.346165 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260619 02:44:15.600139 24645 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:15.600273 24645 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:15.600594 24645 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799), f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
Jun 19 02:44:15 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.2@KRBTEST.COM
Jun 19 02:44:15 dist-test-slave-pdnb krb5kdc[23066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.3@KRBTEST.COM
I20260619 02:44:15.610792 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.600856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:46091 (local address 127.7.137.3:45799)
0619 02:44:15.601055 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:15.601065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:15.601093 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:44:15.601698 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:15.601700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:15.601741 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:15.601881 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:15.602655 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.603497 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.604469 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.604910 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.605040 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:15.607686 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:15.607696 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:15.607696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:15.607705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:15.609233 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609616 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:15.609619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:15.609620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:15.609716 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609991 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:15.609992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:15.609992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:15.610203 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:15.610289 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:15.610565 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 02:44:15.610680 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":60,"threads_started":1}
I20260619 02:44:15.610783 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.600756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:45799 (local address 127.7.137.1:46091)
0619 02:44:15.600812 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:15.600824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:15.601058 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:15.601914 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:15.601917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:15.602318 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:15.602527 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:15.602533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.603601 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.603602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:15.604360 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:15.604362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.605002 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.605003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:15.605117 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:15.605765 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:15.605769 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:15.607589 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:15.609311 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:15.609312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:15.609542 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:15.609787 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:15.609788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:15.609906 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:15.610281 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:15.610288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:15.610411 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 02:44:15.610517 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:44:15.610690 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":67}
I20260619 02:44:15.610837 24648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.600908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:45005 (local address 127.7.137.2:44703)
0619 02:44:15.601056 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:15.601067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:15.601099 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:44:15.601684 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:15.601687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:15.601743 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:15.601880 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:15.602655 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.603493 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.604453 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.604912 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.605031 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:15.607238 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:15.607242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:15.607243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:15.607255 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:15.609164 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609648 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:15.609650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:15.609651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:15.609725 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609978 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:15.609979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:15.609979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:15.610204 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:15.610304 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:15.610583 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 02:44:15.610721 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260619 02:44:15.610921 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:15.600771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:44703 (local address 127.7.137.1:45005)
0619 02:44:15.600899 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:15.600908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:15.600960 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:15.601914 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:15.601917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:15.602319 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:15.602527 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:15.602533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.603612 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.603613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:15.604358 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:15.604360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:15.605003 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:15.605004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:15.605117 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:15.605696 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:15.605705 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:15.607127 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:15.609246 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:15.609255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:15.609558 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:15.609805 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:15.609807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:15.609807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:15.609908 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:15.610288 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:15.610288 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:15.610415 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 02:44:15.610518 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:44:15.610777 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":132,"thread_start_us":64,"threads_started":1}
W20260619 02:44:15.612397 24261 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
W20260619 02:44:15.612488 24258 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:44:15.612545 24258 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf; no voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280
I20260619 02:44:15.612643 24645 raft_consensus.cc:2726] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:44:16.159257 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:16.159031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:47867)
0619 02:44:16.159178 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:16.159213 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260619 02:44:16.219174 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:16.218949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:48327)
0619 02:44:16.219068 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:16.219106 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260619 02:44:16.346843 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:16.346654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:49905)
0619 02:44:16.346774 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:16.346809 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260619 02:44:17.159904 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:17.159702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:50535)
0619 02:44:17.159823 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:17.159859 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260619 02:44:17.219883 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:17.219640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:41037)
0619 02:44:17.219805 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:17.219842 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":66,"threads_started":1}
I20260619 02:44:17.346813 24654 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:17.346906 24654 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:17.347087 24654 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799), f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
W20260619 02:44:17.347371 24258 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
W20260619 02:44:17.347496 24261 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:44:17.347543 24261 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf; no voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280
I20260619 02:44:17.347632 24654 raft_consensus.cc:2726] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:44:17.347682 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:17.347419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:33881)
0619 02:44:17.347606 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:17.347641 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260619 02:44:18.160620 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:18.160431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:60639)
0619 02:44:18.160547 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:18.160575 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260619 02:44:18.220477 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:18.220284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:58231)
0619 02:44:18.220407 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:18.220440 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260619 02:44:18.348369 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:18.348166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:41521)
0619 02:44:18.348302 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:18.348331 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260619 02:44:19.161320 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:19.161092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:56785)
0619 02:44:19.161236 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:19.161273 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260619 02:44:19.221138 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:19.220947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:37991)
0619 02:44:19.221068 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:19.221103 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 02:44:19.349023 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:19.348819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:55539)
0619 02:44:19.348940 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:19.348979 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260619 02:44:19.576117 24662 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:19.576215 24662 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:19.576431 24662 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799), f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
W20260619 02:44:19.576843 24258 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
W20260619 02:44:19.576908 24261 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:44:19.576943 24261 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf; no voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280
I20260619 02:44:19.577021 24662 raft_consensus.cc:2726] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:44:20.162022 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:20.161751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:49011)
0619 02:44:20.161933 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:20.161972 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260619 02:44:20.221707 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:20.221513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:43525)
0619 02:44:20.221634 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:20.221667 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260619 02:44:20.349689 24665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:20.349450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:54047)
0619 02:44:20.349611 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:20.349648 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260619 02:44:21.162779 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:21.162559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:37465)
0619 02:44:21.162688 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:21.162731 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260619 02:44:21.222363 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:21.222153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:46401)
0619 02:44:21.222286 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:21.222322 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260619 02:44:21.350427 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:21.350172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:43103)
0619 02:44:21.350341 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:21.350382 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260619 02:44:21.961350 24669 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:21.961447 24669 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:21.961668 24669 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799), f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
W20260619 02:44:21.962095 24258 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
W20260619 02:44:21.962157 24261 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:44:21.962190 24261 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf; no voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280
I20260619 02:44:21.962284 24669 raft_consensus.cc:2726] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:44:22.163471 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:22.163289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:42005)
0619 02:44:22.163398 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:22.163426 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260619 02:44:22.223043 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:22.222836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:52207)
0619 02:44:22.222970 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:22.223002 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260619 02:44:22.351127 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:22.350883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:38761)
0619 02:44:22.351030 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:22.351067 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260619 02:44:23.164152 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:23.163946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:46683)
0619 02:44:23.164069 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:23.164106 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260619 02:44:23.223809 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:23.223594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:58083)
0619 02:44:23.223722 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:23.223754 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260619 02:44:23.351783 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:23.351583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:53503)
0619 02:44:23.351700 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:23.351739 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 02:44:24.164978 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:24.164702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:50637)
0619 02:44:24.164895 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:24.164932 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":124,"threads_started":1}
I20260619 02:44:24.224472 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:24.224255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:43157)
0619 02:44:24.224396 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:24.224430 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260619 02:44:24.352454 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:24.352223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:56449)
0619 02:44:24.352353 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:24.352402 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260619 02:44:25.165719 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:25.165510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:47927)
0619 02:44:25.165637 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:25.165673 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260619 02:44:25.225123 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:25.224915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:54431)
0619 02:44:25.225038 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:25.225072 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260619 02:44:25.353143 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:25.352926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:36801)
0619 02:44:25.353053 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:25.353093 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260619 02:44:26.166414 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:26.166202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:35167)
0619 02:44:26.166331 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:26.166368 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260619 02:44:26.225744 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:26.225543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:50697)
0619 02:44:26.225664 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:26.225700 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260619 02:44:26.353818 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:26.353589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:45303)
0619 02:44:26.353731 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:26.353771 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260619 02:44:27.167109 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:27.166905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:55689)
0619 02:44:27.167027 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:27.167059 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260619 02:44:27.226375 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:27.226168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:57001)
0619 02:44:27.226297 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:27.226330 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260619 02:44:27.354498 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:27.354273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:51567)
0619 02:44:27.354409 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:27.354449 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":55,"threads_started":1}
I20260619 02:44:28.167737 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:28.167515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:40147)
0619 02:44:28.167657 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:28.167692 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260619 02:44:28.227079 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:28.226831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:57553)
0619 02:44:28.226993 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:28.227037 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260619 02:44:28.355291 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:28.355068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:42461)
0619 02:44:28.355212 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:28.355246 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260619 02:44:28.987643 24691 raft_consensus.cc:482] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:28.987787 24691 raft_consensus.cc:504] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829617fadb71476c8d5def4a4e0b7176" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 45799 } } peers { permanent_uuid: "f3ac516c9de84b1db3e8823252397280" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 44703 } } peers { permanent_uuid: "1432e3760ec64923a684fee743d55fdf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 44883 } }
I20260619 02:44:28.987991 24691 leader_election.cc:288] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799), f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703)
W20260619 02:44:28.988409 24258 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 829617fadb71476c8d5def4a4e0b7176 (127.7.137.3:45799): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
W20260619 02:44:28.988474 24261 leader_election.cc:341] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3ac516c9de84b1db3e8823252397280 (127.7.137.2:44703): Not found: Tablet not found: 20013b2a51fd4a03a1dbd5a666c5643d
I20260619 02:44:28.988507 24261 leader_election.cc:302] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1432e3760ec64923a684fee743d55fdf; no voters: 829617fadb71476c8d5def4a4e0b7176, f3ac516c9de84b1db3e8823252397280
I20260619 02:44:28.988576 24691 raft_consensus.cc:2726] T 20013b2a51fd4a03a1dbd5a666c5643d P 1432e3760ec64923a684fee743d55fdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:44:29.168498 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:29.168292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.1:32807)
0619 02:44:29.168419 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:29.168451 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260619 02:44:29.227844 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:29.227622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.2:60113)
0619 02:44:29.227765 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:29.227800 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260619 02:44:29.355995 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:29.355802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:45327 (local address 127.7.137.3:56113)
0619 02:44:29.355920 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:29.355951 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.62:45327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260619 02:44:29.360225 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24208
I20260619 02:44:29.364687 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24375
I20260619 02:44:29.369063 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24509
2026-06-19T02:44:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58522 ms)
[----------] 30 tests from SecurityITest (229775 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:29 dist-test-slave-pdnb krb5kdc[24697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:29 dist-test-slave-pdnb krb5kdc[24697](info): set up 2 sockets
Jun 19 02:44:29 dist-test-slave-pdnb krb5kdc[24697](info): commencing operation
krb5kdc: starting...
W20260619 02:44:30.452307 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:30.565747 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41063
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:41063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:30.644523 24710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:30.644774 24710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:30.644814 24710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:30.645941 24710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:30.646010 24710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:30.646034 24710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:30.646063 24710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:30.646095 24710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:30.647600 24710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:41063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:41063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:30.647866 24710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:30.648138 24710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:30.653318 24716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:30.653344 24718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:30.653398 24710 server_base.cc:1034] running on GCE node
W20260619 02:44:30.653332 24715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:30.653797 24710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:30.654080 24710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:30.655241 24710 hybrid_clock.cc:648] HybridClock initialized: now 1781837070655232 us; error 48 us; skew 500 ppm
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:30.657658 24710 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:30.658098 24710 webserver.cc:460] Webserver started at http://127.7.137.62:45823/ using document root <none> and password file <none>
I20260619 02:44:30.658337 24710 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:30.658404 24710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:30.658550 24710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:30.659595 24710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "b27f42bb58c744b09713ea6096714440"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "9b89d42670c7beb53a546d1247704442"
server_key_iv: "6fc0289941a003d6d628213b899f0581"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.659950 24710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "b27f42bb58c744b09713ea6096714440"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "9b89d42670c7beb53a546d1247704442"
server_key_iv: "6fc0289941a003d6d628213b899f0581"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.661302 24710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:30.661985 24725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:30.662147 24710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:30.662250 24710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "b27f42bb58c744b09713ea6096714440"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "9b89d42670c7beb53a546d1247704442"
server_key_iv: "6fc0289941a003d6d628213b899f0581"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.662333 24710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:30.676469 24710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:30.678475 24710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:30.678637 24710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:30.682468 24710 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:41063
I20260619 02:44:30.682503 24777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:41063 every 8 connection(s)
I20260619 02:44:30.682997 24710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:30.683718 24778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:30.686182 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: Bootstrap starting.
I20260619 02:44:30.686750 24778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:30.687038 24778 log.cc:824] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:30.687819 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: No bootstrap required, opened a new log
I20260619 02:44:30.689713 24778 raft_consensus.cc:348] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } }
I20260619 02:44:30.689898 24778 raft_consensus.cc:374] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:30.690003 24778 raft_consensus.cc:729] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27f42bb58c744b09713ea6096714440, State: Initialized, Role: FOLLOWER
I20260619 02:44:30.690145 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24710
I20260619 02:44:30.690241 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:30.690188 24778 consensus_queue.cc:260] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } }
I20260619 02:44:30.690299 24778 raft_consensus.cc:388] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:30.690342 24778 raft_consensus.cc:482] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:30.690394 24778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:30.690423 7716 external_mini_cluster.cc:1428] Setting key b1a3fe0c5aed949f107e47386d5a6e68
I20260619 02:44:30.691823 24778 raft_consensus.cc:504] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } }
I20260619 02:44:30.691919 24778 leader_election.cc:302] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b27f42bb58c744b09713ea6096714440; no voters:
I20260619 02:44:30.692130 24778 leader_election.cc:288] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:30.692155 24783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:30.692279 24783 raft_consensus.cc:686] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [term 1 LEADER]: Becoming Leader. State: Replica: b27f42bb58c744b09713ea6096714440, State: Running, Role: LEADER
I20260619 02:44:30.692404 24778 sys_catalog.cc:564] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:30.692425 24783 consensus_queue.cc:237] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } }
I20260619 02:44:30.692883 24783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b27f42bb58c744b09713ea6096714440. Latest consensus state: current_term: 1 leader_uuid: "b27f42bb58c744b09713ea6096714440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } } }
I20260619 02:44:30.693193 24783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:30.693111 24785 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b27f42bb58c744b09713ea6096714440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27f42bb58c744b09713ea6096714440" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 41063 } } }
I20260619 02:44:30.693599 24792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:30.693638 24785 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:30.694236 24792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:30.695870 24792 catalog_manager.cc:1347] Generated new cluster ID: eec4802c3669484999d0e6b43e9552e6
I20260619 02:44:30.695921 24792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:30.702807 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:30.690899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.7.137.62:41063)
0619 02:44:30.691178 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:30.691190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:30.691244 (+ 54us) server_negotiation.cc:407] Connection header received
0619 02:44:30.691398 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:30.691408 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:30.691475 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:30.691640 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:30.692365 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.693791 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.694964 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.695476 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.695591 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:30.698774 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:30.698779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:30.698779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:30.698792 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:30.700980 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:30.701371 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:30.701374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:30.701375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:30.701513 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:30.701716 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:30.701717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:30.701717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:30.702048 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:30.702246 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:30.702326 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:44:30.702538 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":147,"threads_started":1}
I20260619 02:44:30.706516 24792 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:30.707254 24792 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:30.716549 24792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: Generated new TSK 0
I20260619 02:44:30.716754 24792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:44:30.747540 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:30.833959 24804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:30.834146 24804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:30.834169 24804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:30.835297 24804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:30.835341 24804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:30.835367 24804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:30.836892 24804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:30.837118 24804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:30.837363 24804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:30.842459 24812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:30.842478 24809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:30.842490 24804 server_base.cc:1034] running on GCE node
W20260619 02:44:30.842461 24810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:30.843050 24804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:30.843259 24804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:30.844483 24804 hybrid_clock.cc:648] HybridClock initialized: now 1781837070844460 us; error 41 us; skew 500 ppm
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:30.847182 24804 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:44:30.847574 24804 webserver.cc:460] Webserver started at http://127.7.137.1:40951/ using document root <none> and password file <none>
I20260619 02:44:30.847770 24804 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:30.847810 24804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:30.847898 24804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:30.848954 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "fc3efa963be14a05a51549a18b1cff47"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "dd9f308ed145490e27db8242a4d57d09"
server_key_iv: "b5228575f34174e684e26e3038128f5a"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.849275 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "fc3efa963be14a05a51549a18b1cff47"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "dd9f308ed145490e27db8242a4d57d09"
server_key_iv: "b5228575f34174e684e26e3038128f5a"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.850632 24804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:30.851342 24819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:30.851578 24804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:30.851666 24804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "fc3efa963be14a05a51549a18b1cff47"
format_stamp: "Formatted at 2026-06-19 02:44:30 on dist-test-slave-pdnb"
server_key: "dd9f308ed145490e27db8242a4d57d09"
server_key_iv: "b5228575f34174e684e26e3038128f5a"
server_key_version: "encryptionkey@0"
I20260619 02:44:30.851724 24804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:30.868801 24804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:30.871968 24804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:30.872098 24804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:30.872310 24804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:30.872593 24804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:30.872627 24804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:30.872650 24804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:30.872671 24804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:30.878938 24804 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45159
I20260619 02:44:30.879030 24932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45159 every 8 connection(s)
I20260619 02:44:30.879356 24804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:44:30.884173 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24804
I20260619 02:44:30.884290 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:30.884501 7716 external_mini_cluster.cc:1428] Setting key f7b51aa4fb6f63240df1a8688eff5723
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:30.889256 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:30.880199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53977 (local address 127.7.137.62:41063)
0619 02:44:30.880265 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:30.880267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:30.880549 (+ 282us) server_negotiation.cc:407] Connection header received
0619 02:44:30.881052 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:30.881053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:30.881067 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:30.881131 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:30.882045 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.882598 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.883580 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.883785 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.883840 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:30.886257 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:30.886259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:30.886259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:30.886266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:30.887868 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:30.888261 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:30.888262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:30.888263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:30.888318 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:30.888576 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:30.888577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:30.888577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:30.888697 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:30.888811 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:30.889080 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 02:44:30.889160 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:44:30.889459 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:30.880176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41063 (local address 127.7.137.1:53977)
0619 02:44:30.880359 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:30.880413 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:30.880963 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:30.881210 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:30.881213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:30.881769 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:30.881960 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:30.881964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.882685 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.882685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:30.883489 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:30.883491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:30.883859 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:30.883860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:30.883975 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:30.884566 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:30.884577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:30.886161 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:30.887955 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:30.887962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:30.887964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:30.888186 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:30.888395 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:30.888396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:30.888397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:30.888509 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:30.888803 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:30.888810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:30.888949 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:44:30.889059 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:44:30.889223 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260619 02:44:30.889925 24933 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41063
I20260619 02:44:30.890045 24933 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:30.890277 24933 heartbeater.cc:511] Master 127.7.137.62:41063 requested a full tablet report, sending...
I20260619 02:44:30.890810 24742 ts_manager.cc:194] Registered new tserver with Master: fc3efa963be14a05a51549a18b1cff47 (127.7.137.1:45159)
I20260619 02:44:30.891438 24742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:53977
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:44:30.910858 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:30.991830 24938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:30.992033 24938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:30.992057 24938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:30.993408 24938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:30.993448 24938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:30.993475 24938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:30.994988 24938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:30.995177 24938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:30.995348 24938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:31.000190 24946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.000295 24938 server_base.cc:1034] running on GCE node
W20260619 02:44:31.000185 24944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.000222 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.000677 24938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:31.000954 24938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:31.002113 24938 hybrid_clock.cc:648] HybridClock initialized: now 1781837071002103 us; error 30 us; skew 500 ppm
Jun 19 02:44:30 dist-test-slave-pdnb krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:31.004801 24938 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:44:31.005263 24938 webserver.cc:460] Webserver started at http://127.7.137.2:37471/ using document root <none> and password file <none>
I20260619 02:44:31.005519 24938 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:31.005594 24938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:31.005749 24938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:31.006865 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "278b760e4c17453489f51a2d89d93729"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "a803c0a349375a723382e1e4d87bb109"
server_key_iv: "a74317a78fed83b782710da877d1aa8e"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.007251 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "278b760e4c17453489f51a2d89d93729"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "a803c0a349375a723382e1e4d87bb109"
server_key_iv: "a74317a78fed83b782710da877d1aa8e"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.008632 24938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:31.009411 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.009580 24938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:31.009682 24938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "278b760e4c17453489f51a2d89d93729"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "a803c0a349375a723382e1e4d87bb109"
server_key_iv: "a74317a78fed83b782710da877d1aa8e"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.009768 24938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:31.034950 24938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:31.036949 24938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:31.037106 24938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:31.037338 24938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:31.037652 24938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:31.037707 24938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.037750 24938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:31.037777 24938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.043613 24938 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:34169
I20260619 02:44:31.043656 25066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:34169 every 8 connection(s)
I20260619 02:44:31.043977 24938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:31.045543 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 24938
I20260619 02:44:31.045656 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:31.045830 7716 external_mini_cluster.cc:1428] Setting key 8229ea89631d705819a8cbcef2519b23
Jun 19 02:44:31 dist-test-slave-pdnb krb5kdc[24697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837070, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:44:31.054188 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.044662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:37739 (local address 127.7.137.62:41063)
0619 02:44:31.044766 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.044768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.044962 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:44:31.045477 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.045478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.045492 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.045554 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.046690 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.047188 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.048614 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.048826 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.048885 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.051301 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.051302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.051302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:31.051310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.052796 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.053215 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.053217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.053217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.053278 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.053525 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.053526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.053526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.053638 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.053712 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.054010 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.054096 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:44:31.054369 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.044628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41063 (local address 127.7.137.2:37739)
0619 02:44:31.044827 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.044869 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.045363 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.045628 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.045632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.046337 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:31.046600 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.046606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.047287 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.047288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.048506 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.048509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.048911 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.048912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.049075 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.049580 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:31.049610 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:31.051201 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:31.052888 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:31.052895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:31.052897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:31.053126 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:31.053344 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:31.053345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:31.053345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:31.053457 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:31.053704 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:31.053710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:31.053867 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.053988 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.054142 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":128,"threads_started":1}
I20260619 02:44:31.054864 25067 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41063
I20260619 02:44:31.054963 25067 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:31.055135 25067 heartbeater.cc:511] Master 127.7.137.62:41063 requested a full tablet report, sending...
I20260619 02:44:31.055507 24742 ts_manager.cc:194] Registered new tserver with Master: 278b760e4c17453489f51a2d89d93729 (127.7.137.2:34169)
I20260619 02:44:31.055945 24742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:37739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:44:31.069495 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:31.149956 25072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:31.150173 25072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:31.150197 25072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:31.151252 25072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:31.151288 25072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:31.151312 25072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:31.152649 25072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:31.152842 25072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:31.153009 25072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:31.157807 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.157830 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.157822 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.158066 25072 server_base.cc:1034] running on GCE node
I20260619 02:44:31.158341 25072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:31.158562 25072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:31.159714 25072 hybrid_clock.cc:648] HybridClock initialized: now 1781837071159709 us; error 49 us; skew 500 ppm
Jun 19 02:44:31 dist-test-slave-pdnb krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837071, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:31.162174 25072 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:44:31.162585 25072 webserver.cc:460] Webserver started at http://127.7.137.3:37189/ using document root <none> and password file <none>
I20260619 02:44:31.162803 25072 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:31.162866 25072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:31.163008 25072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:31.163990 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "d034c15c073f4f268cbdf46cd1be7425"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "8813787b0e49cef3dd5c121ba395d2a1"
server_key_iv: "e3b40302a475f5b5006aeeceadbe9546"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.164343 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "d034c15c073f4f268cbdf46cd1be7425"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "8813787b0e49cef3dd5c121ba395d2a1"
server_key_iv: "e3b40302a475f5b5006aeeceadbe9546"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.165654 25072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:31.166401 25087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.166548 25072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.166640 25072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "d034c15c073f4f268cbdf46cd1be7425"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "8813787b0e49cef3dd5c121ba395d2a1"
server_key_iv: "e3b40302a475f5b5006aeeceadbe9546"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.166711 25072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:31.189599 25072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:31.191867 25072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:31.191982 25072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:31.192279 25072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:31.192548 25072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:31.192584 25072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.192605 25072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:31.192618 25072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.198524 25072 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:32955
I20260619 02:44:31.198562 25200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:32955 every 8 connection(s)
I20260619 02:44:31.198886 25072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:31.204260 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25072
I20260619 02:44:31.204466 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:31.204695 7716 external_mini_cluster.cc:1428] Setting key a23952512463e4d9f776383189bff88b
Jun 19 02:44:31 dist-test-slave-pdnb krb5kdc[24697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837071, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:31.208086 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.199574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:42951 (local address 127.7.137.62:41063)
0619 02:44:31.199640 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.199642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.199840 (+ 198us) server_negotiation.cc:407] Connection header received
0619 02:44:31.200368 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.200369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.200382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.200433 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.201393 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.201887 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.203063 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.203261 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.203352 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.205760 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.205761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.205761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:31.205767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.207028 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.207373 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.207374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.207374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.207420 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.207592 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.207593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.207593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.207688 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.207738 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.207967 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.208025 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:44:31.208334 25203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.199568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:41063 (local address 127.7.137.3:42951)
0619 02:44:31.199728 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.199762 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.200257 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.200499 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.200507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.201113 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:31.201299 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.201304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.201977 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.201978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.202976 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.202979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.203372 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.203373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.203485 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.203930 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:31.203938 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:31.205696 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:31.207101 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:31.207106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:31.207108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:31.207310 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:31.207458 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:31.207459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:31.207459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:31.207545 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:31.207744 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:31.207748 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:31.207863 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.207942 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.208065 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260619 02:44:31.208789 25201 heartbeater.cc:347] Connected to a master server at 127.7.137.62:41063
I20260619 02:44:31.208892 25201 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:31.209079 25201 heartbeater.cc:511] Master 127.7.137.62:41063 requested a full tablet report, sending...
I20260619 02:44:31.209434 24742 ts_manager.cc:194] Registered new tserver with Master: d034c15c073f4f268cbdf46cd1be7425 (127.7.137.3:32955)
I20260619 02:44:31.209810 24742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:42951
I20260619 02:44:31.216557 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:31.223610 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.218017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.7.137.62:41063)
0619 02:44:31.218057 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.218059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.218114 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:44:31.218166 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.218167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.218175 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.218212 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.218764 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.219215 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.219933 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.220098 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.220154 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.221251 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.221252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.221252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:31.221258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.222579 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.222900 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.222901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.222901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.222949 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:31.223076 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:31.223077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:31.223077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:31.223205 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.223279 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.223472 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.223528 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:31.225121 24742 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:31.226606 24742 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:31.233489 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.228896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45159 (local address 127.0.0.1:47480)
0619 02:44:31.228995 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.229040 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.229123 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.229631 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.229634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.229636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:31.229712 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.229714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.231075 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.231076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.232167 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.232169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.233179 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.233253 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.233266 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.233335 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.233385 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":14,"thread_start_us":54,"threads_started":1}
I20260619 02:44:31.233493 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.229254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47480 (local address 127.7.137.1:45159)
0619 02:44:31.229432 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.229438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.229450 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:44:31.229495 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.229497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.229513 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.229785 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:31.229875 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.230972 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.232257 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.233065 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233176 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.233225 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.233360 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.233403 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260619 02:44:31.233721 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.229238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.7.137.2:34169)
0619 02:44:31.229413 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.229418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.229776 (+ 358us) server_negotiation.cc:407] Connection header received
0619 02:44:31.229818 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.229820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.229836 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.229915 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:31.230100 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.230774 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.232258 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.233066 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233140 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.233200 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.233538 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.233591 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260619 02:44:31.233500 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.229112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:34169 (local address 127.0.0.1:38836)
0619 02:44:31.229612 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.229620 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.229666 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.229994 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.229994 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.229996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:31.230027 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.230028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.230872 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.230873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.232167 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.232169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.233234 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.233298 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.233302 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.233373 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.233412 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260619 02:44:31.234285 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.228910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:32955 (local address 127.0.0.1:50118)
0619 02:44:31.229458 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.229469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.229528 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.229769 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.229770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.229772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:31.229818 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.229820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.232268 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.232269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.233057 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.233059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.234049 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.234050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.234131 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.234135 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.234184 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.234226 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":84,"threads_started":1}
I20260619 02:44:31.234506 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.229066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.7.137.3:32955)
0619 02:44:31.229460 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.229463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.229519 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:44:31.229596 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.229598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.229614 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.229692 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:31.229897 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.230983 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233140 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.233887 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.233958 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.234014 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.234386 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.234431 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":100,"threads_started":1}
I20260619 02:44:31.234586 24867 tablet_service.cc:1467] Processing CreateTablet for tablet 783836d2081b443ba4c032bf621dc7eb (DEFAULT_TABLE table=test-table [id=4d63f64113874218ac1e9157971021ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:31.234637 25001 tablet_service.cc:1467] Processing CreateTablet for tablet 783836d2081b443ba4c032bf621dc7eb (DEFAULT_TABLE table=test-table [id=4d63f64113874218ac1e9157971021ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:31.234928 24867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 783836d2081b443ba4c032bf621dc7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:31.234928 25001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 783836d2081b443ba4c032bf621dc7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:31.235030 25135 tablet_service.cc:1467] Processing CreateTablet for tablet 783836d2081b443ba4c032bf621dc7eb (DEFAULT_TABLE table=test-table [id=4d63f64113874218ac1e9157971021ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:31.235255 25135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 783836d2081b443ba4c032bf621dc7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:31.237624 25218 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Bootstrap starting.
I20260619 02:44:31.238175 25220 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Bootstrap starting.
I20260619 02:44:31.238287 25218 tablet_bootstrap.cc:654] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:31.238847 25218 log.cc:824] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:31.238873 25220 tablet_bootstrap.cc:654] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:31.239437 25220 log.cc:824] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:31.239615 25218 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: No bootstrap required, opened a new log
I20260619 02:44:31.239691 25218 ts_tablet_manager.cc:1397] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:31.239785 25219 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Bootstrap starting.
I20260619 02:44:31.240124 25220 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: No bootstrap required, opened a new log
I20260619 02:44:31.240208 25220 ts_tablet_manager.cc:1397] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:31.240397 25219 tablet_bootstrap.cc:654] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:31.241042 25219 log.cc:824] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:31.241816 25219 tablet_bootstrap.cc:492] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: No bootstrap required, opened a new log
I20260619 02:44:31.241884 25219 ts_tablet_manager.cc:1397] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:31.241860 25218 raft_consensus.cc:348] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.242046 25218 raft_consensus.cc:374] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:31.242089 25218 raft_consensus.cc:729] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d034c15c073f4f268cbdf46cd1be7425, State: Initialized, Role: FOLLOWER
I20260619 02:44:31.242060 25220 raft_consensus.cc:348] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.242224 25220 raft_consensus.cc:374] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:31.242224 25218 consensus_queue.cc:260] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.242264 25220 raft_consensus.cc:729] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3efa963be14a05a51549a18b1cff47, State: Initialized, Role: FOLLOWER
I20260619 02:44:31.242374 25220 consensus_queue.cc:260] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.242571 25218 ts_tablet_manager.cc:1428] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:31.242625 25201 heartbeater.cc:503] Master 127.7.137.62:41063 was elected leader, sending a full tablet report...
I20260619 02:44:31.242671 25220 ts_tablet_manager.cc:1428] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 02:44:31.242802 24933 heartbeater.cc:503] Master 127.7.137.62:41063 was elected leader, sending a full tablet report...
I20260619 02:44:31.243953 25219 raft_consensus.cc:348] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.244084 25219 raft_consensus.cc:374] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:31.244105 25219 raft_consensus.cc:729] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278b760e4c17453489f51a2d89d93729, State: Initialized, Role: FOLLOWER
I20260619 02:44:31.244206 25219 consensus_queue.cc:260] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.244426 25219 ts_tablet_manager.cc:1428] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:31.244490 25067 heartbeater.cc:503] Master 127.7.137.62:41063 was elected leader, sending a full tablet report...
W20260619 02:44:31.294503 25068 tablet.cc:2369] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:44:31.380261 24934 tablet.cc:2369] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:31.445165 25225 raft_consensus.cc:482] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:31.445289 25225 raft_consensus.cc:504] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.445681 25225 leader_election.cc:288] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d034c15c073f4f268cbdf46cd1be7425 (127.7.137.3:32955), 278b760e4c17453489f51a2d89d93729 (127.7.137.2:34169)
I20260619 02:44:31.448627 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.446024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:38139 (local address 127.7.137.3:32955)
0619 02:44:31.446092 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.446095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.446176 (+ 81us) server_negotiation.cc:407] Connection header received
0619 02:44:31.446268 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.446269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.446283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.446351 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:31.446512 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.446982 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.447913 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.448284 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.448366 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.448377 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.448527 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.448564 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:44:31.448652 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.445915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:32955 (local address 127.7.137.1:38139)
0619 02:44:31.446088 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.446101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.446196 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.446355 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.446356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.446357 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:31.446441 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.446443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.447064 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.447065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.447823 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.447825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.448366 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.448367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.448436 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.448442 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.448519 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.448559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260619 02:44:31.448918 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.446152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:34169 (local address 127.7.137.1:53693)
0619 02:44:31.446268 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.446279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.446336 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.446495 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.446496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.446498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:31.446599 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.446601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.447253 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.447254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.447992 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.447993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.448581 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.448582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.448654 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.448657 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.448788 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.448840 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":53,"threads_started":1}
I20260619 02:44:31.448939 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.446248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53693 (local address 127.7.137.2:34169)
0619 02:44:31.446294 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.446297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.446325 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:44:31.446410 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.446411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.446421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.446492 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:31.446681 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.447160 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.448087 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.448488 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.448565 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.448575 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.448835 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.448880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:44:31.448942 25155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "783836d2081b443ba4c032bf621dc7eb" candidate_uuid: "fc3efa963be14a05a51549a18b1cff47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d034c15c073f4f268cbdf46cd1be7425" is_pre_election: true
I20260619 02:44:31.449090 25155 raft_consensus.cc:2454] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3efa963be14a05a51549a18b1cff47 in term 0.
I20260619 02:44:31.449257 25021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "783836d2081b443ba4c032bf621dc7eb" candidate_uuid: "fc3efa963be14a05a51549a18b1cff47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278b760e4c17453489f51a2d89d93729" is_pre_election: true
I20260619 02:44:31.449345 24822 leader_election.cc:302] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d034c15c073f4f268cbdf46cd1be7425, fc3efa963be14a05a51549a18b1cff47; no voters:
W20260619 02:44:31.449388 25202 tablet.cc:2369] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:31.449411 25021 raft_consensus.cc:2454] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3efa963be14a05a51549a18b1cff47 in term 0.
I20260619 02:44:31.449463 25225 raft_consensus.cc:2781] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:31.449513 25225 raft_consensus.cc:482] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:31.449541 25225 raft_consensus.cc:3037] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:31.450290 25225 raft_consensus.cc:504] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.450393 25225 leader_election.cc:288] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [CANDIDATE]: Term 1 election: Requested vote from peers d034c15c073f4f268cbdf46cd1be7425 (127.7.137.3:32955), 278b760e4c17453489f51a2d89d93729 (127.7.137.2:34169)
I20260619 02:44:31.450613 25155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "783836d2081b443ba4c032bf621dc7eb" candidate_uuid: "fc3efa963be14a05a51549a18b1cff47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d034c15c073f4f268cbdf46cd1be7425"
I20260619 02:44:31.450609 25021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "783836d2081b443ba4c032bf621dc7eb" candidate_uuid: "fc3efa963be14a05a51549a18b1cff47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278b760e4c17453489f51a2d89d93729"
I20260619 02:44:31.450693 25155 raft_consensus.cc:3037] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:31.450695 25021 raft_consensus.cc:3037] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:31.451577 25021 raft_consensus.cc:2454] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3efa963be14a05a51549a18b1cff47 in term 1.
I20260619 02:44:31.451582 25155 raft_consensus.cc:2454] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3efa963be14a05a51549a18b1cff47 in term 1.
I20260619 02:44:31.451769 24822 leader_election.cc:302] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d034c15c073f4f268cbdf46cd1be7425, fc3efa963be14a05a51549a18b1cff47; no voters:
I20260619 02:44:31.451862 25225 raft_consensus.cc:2781] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:31.452008 25225 raft_consensus.cc:686] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [term 1 LEADER]: Becoming Leader. State: Replica: fc3efa963be14a05a51549a18b1cff47, State: Running, Role: LEADER
I20260619 02:44:31.452144 25225 consensus_queue.cc:237] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } }
I20260619 02:44:31.453003 24741 catalog_manager.cc:5495] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc3efa963be14a05a51549a18b1cff47 (127.7.137.1). New cstate: current_term: 1 leader_uuid: "fc3efa963be14a05a51549a18b1cff47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3efa963be14a05a51549a18b1cff47" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 } health_report { overall_health: UNKNOWN } } }
I20260619 02:44:31.459410 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.457090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.7.137.1:45159)
0619 02:44:31.457146 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.457148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.457156 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:31.457225 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.457226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.457244 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.457329 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:31.457440 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.457899 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.458643 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.458850 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.458932 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.459092 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:31.459196 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:31.459291 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.459310 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.459331 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 02:44:31.461324 25155 raft_consensus.cc:1261] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 1 FOLLOWER]: Refusing update from remote peer fc3efa963be14a05a51549a18b1cff47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:31.461324 25021 raft_consensus.cc:1261] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 1 FOLLOWER]: Refusing update from remote peer fc3efa963be14a05a51549a18b1cff47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:31.461637 25225 consensus_queue.cc:1035] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "278b760e4c17453489f51a2d89d93729" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:31.461808 25229 consensus_queue.cc:1035] T 783836d2081b443ba4c032bf621dc7eb P fc3efa963be14a05a51549a18b1cff47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d034c15c073f4f268cbdf46cd1be7425" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:31.464143 25232 mvcc.cc:205] Tried to move back new op lower bound from 7298404644702228480 to 7298404644668555264. Current Snapshot: MvccSnapshot[applied={T|T < 7298404644702228480}]
I20260619 02:44:31.466801 24742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:31.466898 24742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:31.467592 24742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b27f42bb58c744b09713ea6096714440: Sending DeleteTablet for 3 replicas of tablet 783836d2081b443ba4c032bf621dc7eb
I20260619 02:44:31.467911 24867 tablet_service.cc:1514] Processing DeleteTablet for tablet 783836d2081b443ba4c032bf621dc7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:31 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:47480
I20260619 02:44:31.467933 25001 tablet_service.cc:1514] Processing DeleteTablet for tablet 783836d2081b443ba4c032bf621dc7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:31 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:38836
I20260619 02:44:31.467928 25135 tablet_service.cc:1514] Processing DeleteTablet for tablet 783836d2081b443ba4c032bf621dc7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:31 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:50118
I20260619 02:44:31.468128 25244 tablet_replica.cc:330] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: stopping tablet replica
I20260619 02:44:31.468211 25244 raft_consensus.cc:2229] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:31.468310 25244 raft_consensus.cc:2258] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:31.468334 25243 tablet_replica.cc:330] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: stopping tablet replica
I20260619 02:44:31.468449 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24804
I20260619 02:44:31.468539 25243 raft_consensus.cc:2229] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:31.468683 25243 raft_consensus.cc:2258] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:31.468772 25244 ts_tablet_manager.cc:1905] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:31.469457 25243 ts_tablet_manager.cc:1905] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:31.470366 25244 ts_tablet_manager.cc:1918] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:31.470453 25244 log.cc:1196] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/783836d2081b443ba4c032bf621dc7eb
I20260619 02:44:31.470721 25244 ts_tablet_manager.cc:1939] T 783836d2081b443ba4c032bf621dc7eb P d034c15c073f4f268cbdf46cd1be7425: Deleting consensus metadata
I20260619 02:44:31.471599 25243 ts_tablet_manager.cc:1918] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:31.471681 25243 log.cc:1196] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/783836d2081b443ba4c032bf621dc7eb
I20260619 02:44:31.471936 25243 ts_tablet_manager.cc:1939] T 783836d2081b443ba4c032bf621dc7eb P 278b760e4c17453489f51a2d89d93729: Deleting consensus metadata
I20260619 02:44:31.472002 24728 catalog_manager.cc:4841] TS d034c15c073f4f268cbdf46cd1be7425 (127.7.137.3:32955): tablet 783836d2081b443ba4c032bf621dc7eb (table test-table [id=4d63f64113874218ac1e9157971021ea]) successfully deleted
I20260619 02:44:31.472328 24728 catalog_manager.cc:4841] TS 278b760e4c17453489f51a2d89d93729 (127.7.137.2:34169): tablet 783836d2081b443ba4c032bf621dc7eb (table test-table [id=4d63f64113874218ac1e9157971021ea]) successfully deleted
W20260619 02:44:31.475371 24727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:45159 (error 108)
I20260619 02:44:31.475637 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24938
I20260619 02:44:31.476294 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.475780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45159 (local address 127.0.0.1:60686)
0619 02:44:31.476192 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.476255 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393}
W20260619 02:44:31.477876 24727 catalog_manager.cc:4584] TS fc3efa963be14a05a51549a18b1cff47 (127.7.137.1:45159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 783836d2081b443ba4c032bf621dc7eb: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45159: connect: Connection refused (error 111)
I20260619 02:44:31.480176 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25072
I20260619 02:44:31.484436 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 24710
2026-06-19T02:44:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2115 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T02:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:31Z Disabled control of system clock
I20260619 02:44:31.511503 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36437
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:31.595611 25249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:31.595825 25249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:31.595846 25249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:31.596959 25249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:31.596997 25249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:31.597010 25249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:31.597023 25249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:31.597034 25249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:31.598315 25249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:31.598505 25249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:31.598742 25249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:31.603612 25249 server_base.cc:1034] running on GCE node
W20260619 02:44:31.603595 25255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.603622 25257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.603585 25254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.603984 25249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:31.604290 25249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:31.605449 25249 hybrid_clock.cc:648] HybridClock initialized: now 1781837071605435 us; error 27 us; skew 500 ppm
I20260619 02:44:31.606563 25249 webserver.cc:460] Webserver started at http://127.7.137.62:40005/ using document root <none> and password file <none>
I20260619 02:44:31.606812 25249 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:31.606880 25249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:31.607034 25249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:31.608067 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "4279a0aec1ce4d168608304f54917be2"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "76e2eae272f5aacece3c7ed1de297f09"
server_key_iv: "be552b44c13e1bc9743a7d4c053e18d8"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.608426 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "4279a0aec1ce4d168608304f54917be2"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "76e2eae272f5aacece3c7ed1de297f09"
server_key_iv: "be552b44c13e1bc9743a7d4c053e18d8"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.609726 25249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:31.610411 25263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.610579 25249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:31.610673 25249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "4279a0aec1ce4d168608304f54917be2"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "76e2eae272f5aacece3c7ed1de297f09"
server_key_iv: "be552b44c13e1bc9743a7d4c053e18d8"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.610754 25249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:31.619266 25249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:31.619549 25249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:31.619691 25249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:31.623620 25249 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36437
I20260619 02:44:31.623651 25315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36437 every 8 connection(s)
I20260619 02:44:31.623975 25249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:31.624976 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:31.627127 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: Bootstrap starting.
I20260619 02:44:31.627710 25316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:31.628000 25316 log.cc:824] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:31.628618 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: No bootstrap required, opened a new log
I20260619 02:44:31.630221 25316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } }
I20260619 02:44:31.630374 25316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:31.630421 25316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4279a0aec1ce4d168608304f54917be2, State: Initialized, Role: FOLLOWER
I20260619 02:44:31.630581 25316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } }
I20260619 02:44:31.630679 25316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:31.630722 25316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:31.630771 25316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:31.631251 25316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } }
I20260619 02:44:31.631361 25316 leader_election.cc:302] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4279a0aec1ce4d168608304f54917be2; no voters:
I20260619 02:44:31.631526 25316 leader_election.cc:288] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:31.631562 25319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:31.631744 25319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [term 1 LEADER]: Becoming Leader. State: Replica: 4279a0aec1ce4d168608304f54917be2, State: Running, Role: LEADER
I20260619 02:44:31.631764 25316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:31.631841 25319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } }
I20260619 02:44:31.632226 25321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4279a0aec1ce4d168608304f54917be2. Latest consensus state: current_term: 1 leader_uuid: "4279a0aec1ce4d168608304f54917be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } } }
I20260619 02:44:31.632305 25320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4279a0aec1ce4d168608304f54917be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4279a0aec1ce4d168608304f54917be2" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36437 } } }
I20260619 02:44:31.632503 25320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:31.632537 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25249
I20260619 02:44:31.632658 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:31.632875 7716 external_mini_cluster.cc:1428] Setting key 5cc8c0c858df80e4e41654fbf4035523
I20260619 02:44:31.632321 25321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:31.633011 25328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:31.633774 25328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:31.635166 25328 catalog_manager.cc:1347] Generated new cluster ID: e3c2fb3ce573412a9fba522993df9ead
I20260619 02:44:31.635238 25328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:31.638804 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.633375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.7.137.62:36437)
0619 02:44:31.633558 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.633568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.633598 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:44:31.633728 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.633735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.633803 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.633942 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.634134 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.635521 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.636586 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.637224 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.637353 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.638018 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.638021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.638022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:31.638239 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.638282 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:31.638296 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.638365 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.638430 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.638605 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260619 02:44:31.651281 25328 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:31.651954 25328 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:31.658906 25328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: Generated new TSK 0
I20260619 02:44:31.659075 25328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:31.663704 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:31.746826 25340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:31.747373 25340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:31.747434 25340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:31.749855 25340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:31.750135 25340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:31.750183 25340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:31.753063 25340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:31.753336 25340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:31.753741 25340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:31.761062 25346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.761564 25348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.761623 25340 server_base.cc:1034] running on GCE node
W20260619 02:44:31.764885 25345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.765240 25340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:31.765591 25340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:31.766716 25340 hybrid_clock.cc:648] HybridClock initialized: now 1781837071766688 us; error 41 us; skew 500 ppm
I20260619 02:44:31.768337 25340 webserver.cc:460] Webserver started at http://127.7.137.1:35157/ using document root <none> and password file <none>
I20260619 02:44:31.768623 25340 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:31.768697 25340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:31.768865 25340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:31.770370 25340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "4af79400d1cdf1b9fddbf6a3c7b2654b"
server_key_iv: "7016b156fe17b1165c276e5ff7b22783"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.770785 25340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "4af79400d1cdf1b9fddbf6a3c7b2654b"
server_key_iv: "7016b156fe17b1165c276e5ff7b22783"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.772143 25340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:31.772895 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.773069 25340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:31.773166 25340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "4af79400d1cdf1b9fddbf6a3c7b2654b"
server_key_iv: "7016b156fe17b1165c276e5ff7b22783"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.773255 25340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:31.790122 25340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:31.790446 25340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:31.790598 25340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:31.790835 25340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:31.791175 25340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:31.791229 25340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.791272 25340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:31.791297 25340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.796916 25340 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:42727
I20260619 02:44:31.796990 25467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:42727 every 8 connection(s)
I20260619 02:44:31.797330 25340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:44:31.798841 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25340
I20260619 02:44:31.798947 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:31.799132 7716 external_mini_cluster.cc:1428] Setting key 60ddbe2afbe7db93d7f1dc89ed984f61
I20260619 02:44:31.800211 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:44:31.803128 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.798059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:46399 (local address 127.7.137.62:36437)
0619 02:44:31.798126 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.798128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.798337 (+ 209us) server_negotiation.cc:407] Connection header received
0619 02:44:31.798886 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.798887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.798903 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.798952 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.799848 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.800489 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.801781 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.801972 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.802030 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.802911 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.802911 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.802912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:31.802915 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.802928 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:31.802931 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.802964 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.803045 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.803070 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:44:31.803403 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.798051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36437 (local address 127.7.137.1:46399)
0619 02:44:31.798220 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.798254 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.798774 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.799061 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.799066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.799076 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:31.799770 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.799777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.800587 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.800587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.801689 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.801692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.802055 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.802057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.802223 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.802750 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:31.802751 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:31.802788 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:31.802817 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:31.802973 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:31.802977 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:31.802980 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.803041 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.803152 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260619 02:44:31.803972 25468 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36437
I20260619 02:44:31.804080 25468 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:31.804262 25468 heartbeater.cc:511] Master 127.7.137.62:36437 requested a full tablet report, sending...
I20260619 02:44:31.804826 25280 ts_manager.cc:194] Registered new tserver with Master: bb64d40c8c174d06a54dab8f2bc8ec8f (127.7.137.1:42727)
I20260619 02:44:31.805441 25280 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.1:46399
W20260619 02:44:31.883093 25471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:31.883294 25471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:31.883315 25471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:31.884471 25471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:31.884512 25471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:31.884537 25471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:31.886040 25471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:31.886279 25471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:31.886566 25471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:31.891482 25479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.891546 25477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:31.891489 25476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:31.891697 25471 server_base.cc:1034] running on GCE node
I20260619 02:44:31.891914 25471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:31.892105 25471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:31.893229 25471 hybrid_clock.cc:648] HybridClock initialized: now 1781837071893214 us; error 28 us; skew 500 ppm
I20260619 02:44:31.894524 25471 webserver.cc:460] Webserver started at http://127.7.137.2:45397/ using document root <none> and password file <none>
I20260619 02:44:31.894765 25471 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:31.894811 25471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:31.894982 25471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:31.896045 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "e9001467de444bbe8488846659e9dad0"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "2e2f4abc1b3c8d88e7bbb8aa76fb4c44"
server_key_iv: "f070e91f35e76c2b5a1410828964f9ca"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.896394 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "e9001467de444bbe8488846659e9dad0"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "2e2f4abc1b3c8d88e7bbb8aa76fb4c44"
server_key_iv: "f070e91f35e76c2b5a1410828964f9ca"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.897758 25471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:31.898429 25485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.898593 25471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:31.898697 25471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "e9001467de444bbe8488846659e9dad0"
format_stamp: "Formatted at 2026-06-19 02:44:31 on dist-test-slave-pdnb"
server_key: "2e2f4abc1b3c8d88e7bbb8aa76fb4c44"
server_key_iv: "f070e91f35e76c2b5a1410828964f9ca"
server_key_version: "encryptionkey@0"
I20260619 02:44:31.898772 25471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:31.908465 25471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:31.908769 25471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:31.908921 25471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:31.909130 25471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:31.909440 25471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:31.909494 25471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.909543 25471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:31.909569 25471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:31.915452 25471 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:37183
I20260619 02:44:31.915495 25598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:37183 every 8 connection(s)
I20260619 02:44:31.915828 25471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:31.920768 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.916496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:43371 (local address 127.7.137.62:36437)
0619 02:44:31.916561 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:31.916563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:31.916793 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:44:31.917269 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:31.917270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:31.917279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:31.917334 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:31.917949 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.918434 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.919386 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.919557 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.919608 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.920452 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:31.920453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:31.920453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:31.920457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:31.920469 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:31.920473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:31.920517 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:31.920644 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:44:31.920668 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:44:31.920957 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:31.916468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36437 (local address 127.7.137.2:43371)
0619 02:44:31.916640 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:31.916679 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:31.917188 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:31.917385 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:31.917388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:31.917396 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:31.917880 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.917884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.918551 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.918552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.919315 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:31.919316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:31.919656 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:31.919657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:31.919814 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:31.920322 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:31.920323 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:31.920349 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:31.920371 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:31.920553 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:31.920559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:31.920562 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:31.920637 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:44:31.920743 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260619 02:44:31.921516 25599 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36437
I20260619 02:44:31.921607 25599 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:31.921789 25599 heartbeater.cc:511] Master 127.7.137.62:36437 requested a full tablet report, sending...
I20260619 02:44:31.922111 25280 ts_manager.cc:194] Registered new tserver with Master: e9001467de444bbe8488846659e9dad0 (127.7.137.2:37183)
I20260619 02:44:31.922472 25280 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.2:43371
I20260619 02:44:31.925757 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25471
I20260619 02:44:31.925836 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:31.925967 7716 external_mini_cluster.cc:1428] Setting key 040560963116a7a2cd9192805cd1666e
I20260619 02:44:31.926803 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--builtin_ntp_servers=127.7.137.20:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:32.009521 25602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:32.009737 25602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:32.009773 25602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:32.010962 25602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:32.011018 25602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:32.011065 25602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:32.012391 25602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:32.012655 25602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:32.013015 25602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:32.017983 25610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:32.017992 25607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:32.018038 25608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:32.018500 25602 server_base.cc:1034] running on GCE node
I20260619 02:44:32.018687 25602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:32.018863 25602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:32.020021 25602 hybrid_clock.cc:648] HybridClock initialized: now 1781837072020009 us; error 26 us; skew 500 ppm
I20260619 02:44:32.021354 25602 webserver.cc:460] Webserver started at http://127.7.137.3:33923/ using document root <none> and password file <none>
I20260619 02:44:32.021564 25602 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:32.021605 25602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:32.021693 25602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:32.022675 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "536447d245f8461fa0694da9d6478900"
format_stamp: "Formatted at 2026-06-19 02:44:32 on dist-test-slave-pdnb"
server_key: "602b24862ccb5fb7cf0b7715a4c4ba41"
server_key_iv: "01fcf8b19854dee2411255983c647db7"
server_key_version: "encryptionkey@0"
I20260619 02:44:32.022959 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "536447d245f8461fa0694da9d6478900"
format_stamp: "Formatted at 2026-06-19 02:44:32 on dist-test-slave-pdnb"
server_key: "602b24862ccb5fb7cf0b7715a4c4ba41"
server_key_iv: "01fcf8b19854dee2411255983c647db7"
server_key_version: "encryptionkey@0"
I20260619 02:44:32.024197 25602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:44:32.025012 25616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:32.025187 25602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:32.025295 25602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "536447d245f8461fa0694da9d6478900"
format_stamp: "Formatted at 2026-06-19 02:44:32 on dist-test-slave-pdnb"
server_key: "602b24862ccb5fb7cf0b7715a4c4ba41"
server_key_iv: "01fcf8b19854dee2411255983c647db7"
server_key_version: "encryptionkey@0"
I20260619 02:44:32.025374 25602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:32.050089 25602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:32.050426 25602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:32.050585 25602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:32.050817 25602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:32.051158 25602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:32.051213 25602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:32.051246 25602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:32.051270 25602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:32.057052 25602 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:44337
I20260619 02:44:32.057113 25729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:44337 every 8 connection(s)
I20260619 02:44:32.057488 25602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:32.061648 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25602
I20260619 02:44:32.061761 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:32.061959 7716 external_mini_cluster.cc:1428] Setting key 4a010eac06e1759de5215d3f8eee906b
I20260619 02:44:32.062973 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.058207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:39387 (local address 127.7.137.62:36437)
0619 02:44:32.058295 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.058297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.058542 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:44:32.058995 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.058996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.059007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.059067 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:32.059746 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.060257 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.061334 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.061669 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.061752 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.062668 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:32.062669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:32.062670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:32.062675 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:32.062690 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:32.062694 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:32.062754 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.062855 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.062895 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:44:32.063211 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.058182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36437 (local address 127.7.137.3:39387)
0619 02:44:32.058396 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.058431 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.058907 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.059124 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.059128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.059137 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:32.059633 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.059637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.060356 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.060357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.061250 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.061252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.061853 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.061854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.061983 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.062522 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:32.062522 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:32.062552 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:32.062580 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:32.062763 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:32.062768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:32.062771 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.062860 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.062958 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260619 02:44:32.063620 25730 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36437
I20260619 02:44:32.063726 25730 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:32.063901 25730 heartbeater.cc:511] Master 127.7.137.62:36437 requested a full tablet report, sending...
I20260619 02:44:32.064280 25280 ts_manager.cc:194] Registered new tserver with Master: 536447d245f8461fa0694da9d6478900 (127.7.137.3:44337)
I20260619 02:44:32.064685 25280 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.3:39387
I20260619 02:44:32.074308 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:32.078450 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.075714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.7.137.62:36437)
0619 02:44:32.075789 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.075791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.075821 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:44:32.075886 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.075887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.075898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.075944 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:32.076096 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.076643 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.077393 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.077594 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.077649 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.078213 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:32.078214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:32.078214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:32.078218 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:32.078230 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:32.078234 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:32.078274 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.078314 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.078337 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:44:32.079905 25280 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:32.081610 25280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:32.088220 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.083849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42727 (local address 127.0.0.1:34654)
0619 02:44:32.083970 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.084009 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.084080 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.084810 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.084811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.084813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:32.084857 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.084859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.086099 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.086100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.087002 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.087005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.087951 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.087952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.088032 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.088048 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.088119 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.088159 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260619 02:44:32.088279 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.084287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.7.137.1:42727)
0619 02:44:32.084447 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.084452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.084463 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:32.084509 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.084511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.084526 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.084701 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:32.084937 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.086029 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.087085 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.087861 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.087937 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.088036 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.088149 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.088193 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260619 02:44:32.089099 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.083871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:44337 (local address 127.0.0.1:41952)
0619 02:44:32.084032 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.084041 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.084079 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.084539 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.084542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.084545 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:32.084618 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.084619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.086043 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.086043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.087164 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.087167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.088870 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.088871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.088934 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.088939 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.088969 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.089010 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":7,"thread_start_us":85,"threads_started":1}
I20260619 02:44:32.089222 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.084113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.7.137.2:37183)
0619 02:44:32.084269 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.084273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.084587 (+ 314us) server_negotiation.cc:407] Connection header received
0619 02:44:32.084708 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.084710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.084746 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.084798 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:32.084966 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.085791 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.087251 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.087909 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.087966 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.088036 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.089062 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.089111 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260619 02:44:32.089172 25402 tablet_service.cc:1467] Processing CreateTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 (DEFAULT_TABLE table=test-table [id=6af2579f43884dbe88a65c777b6b4912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:32.089534 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.084113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.7.137.3:44337)
0619 02:44:32.084337 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.084340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.084347 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:32.084379 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.084380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.084392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.084713 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:32.084790 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.085955 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.088072 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.088771 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.089281 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.089376 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.089404 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.089441 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260619 02:44:32.089546 25402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92730a4f85d4bc7affc5ceafc71f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:32.089876 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.083871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:37183 (local address 127.0.0.1:58302)
0619 02:44:32.084475 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.084486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.084616 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.084880 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.084881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.084882 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:32.084915 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.084917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.085873 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.085875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.087164 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.087167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.088818 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.088819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.088887 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.088892 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.088941 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.088992 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":7,"thread_start_us":107,"threads_started":1}
I20260619 02:44:32.090229 25664 tablet_service.cc:1467] Processing CreateTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 (DEFAULT_TABLE table=test-table [id=6af2579f43884dbe88a65c777b6b4912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:32.090526 25664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92730a4f85d4bc7affc5ceafc71f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:32.090525 25533 tablet_service.cc:1467] Processing CreateTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 (DEFAULT_TABLE table=test-table [id=6af2579f43884dbe88a65c777b6b4912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:32.090852 25533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92730a4f85d4bc7affc5ceafc71f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:32.093088 25744 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: Bootstrap starting.
I20260619 02:44:32.093429 25745 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Bootstrap starting.
I20260619 02:44:32.093883 25744 tablet_bootstrap.cc:654] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:32.094054 25745 tablet_bootstrap.cc:654] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:32.094165 25746 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Bootstrap starting.
I20260619 02:44:32.094215 25744 log.cc:824] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:32.094354 25745 log.cc:824] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:32.094826 25746 tablet_bootstrap.cc:654] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:32.095012 25744 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: No bootstrap required, opened a new log
I20260619 02:44:32.095057 25745 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: No bootstrap required, opened a new log
I20260619 02:44:32.095094 25744 ts_tablet_manager.cc:1397] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:32.095103 25745 ts_tablet_manager.cc:1397] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:32.095132 25746 log.cc:824] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:32.095757 25746 tablet_bootstrap.cc:492] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: No bootstrap required, opened a new log
I20260619 02:44:32.095837 25746 ts_tablet_manager.cc:1397] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:32.097584 25746 raft_consensus.cc:348] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.097743 25746 raft_consensus.cc:374] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:32.097786 25746 raft_consensus.cc:729] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9001467de444bbe8488846659e9dad0, State: Initialized, Role: FOLLOWER
I20260619 02:44:32.097716 25744 raft_consensus.cc:348] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.097703 25745 raft_consensus.cc:348] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.097859 25744 raft_consensus.cc:374] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:32.097859 25745 raft_consensus.cc:374] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:32.097898 25745 raft_consensus.cc:729] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 536447d245f8461fa0694da9d6478900, State: Initialized, Role: FOLLOWER
I20260619 02:44:32.097898 25744 raft_consensus.cc:729] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb64d40c8c174d06a54dab8f2bc8ec8f, State: Initialized, Role: FOLLOWER
I20260619 02:44:32.097919 25746 consensus_queue.cc:260] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.098018 25745 consensus_queue.cc:260] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.098028 25744 consensus_queue.cc:260] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.098177 25746 ts_tablet_manager.cc:1428] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:44:32.098273 25744 ts_tablet_manager.cc:1428] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:44:32.098274 25745 ts_tablet_manager.cc:1428] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:32.098325 25599 heartbeater.cc:503] Master 127.7.137.62:36437 was elected leader, sending a full tablet report...
I20260619 02:44:32.098385 25730 heartbeater.cc:503] Master 127.7.137.62:36437 was elected leader, sending a full tablet report...
I20260619 02:44:32.098575 25468 heartbeater.cc:503] Master 127.7.137.62:36437 was elected leader, sending a full tablet report...
W20260619 02:44:32.166411 25600 tablet.cc:2369] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:32.253798 25751 raft_consensus.cc:482] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:32.253976 25751 raft_consensus.cc:504] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.254403 25751 leader_election.cc:288] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb64d40c8c174d06a54dab8f2bc8ec8f (127.7.137.1:42727), e9001467de444bbe8488846659e9dad0 (127.7.137.2:37183)
I20260619 02:44:32.258394 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.254580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42727 (local address 127.7.137.3:46053)
0619 02:44:32.254663 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.254677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.254726 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.254889 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.254890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.254891 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:32.254991 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.254994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.255972 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.255973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.257371 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.257374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.258111 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.258112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.258199 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.258203 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.258264 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.258303 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":102}
I20260619 02:44:32.258399 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.254681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:46053 (local address 127.7.137.1:42727)
0619 02:44:32.254728 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.254730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.254766 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:44:32.254797 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.254803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.254813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.254876 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:32.255071 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.255861 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.257463 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.258016 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.258092 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.258102 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.258283 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.258320 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:44:32.258399 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.254711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:47941 (local address 127.7.137.2:37183)
0619 02:44:32.254751 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.254753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.254868 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:44:32.254930 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.254931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.254943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.254986 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:32.255148 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.255860 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.257521 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.258029 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.258092 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.258102 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.258283 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.258320 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:32.258406 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.254616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:37183 (local address 127.7.137.3:47941)
0619 02:44:32.254789 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.254797 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:32.254873 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:32.255013 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:32.255013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:32.255015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:32.255073 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.255074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.255972 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.255973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.257371 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:32.257374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.258111 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.258112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:32.258199 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.258203 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:32.258271 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:44:32.258301 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":54,"thread_start_us":102,"threads_started":1}
I20260619 02:44:32.258744 25422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92730a4f85d4bc7affc5ceafc71f459" candidate_uuid: "536447d245f8461fa0694da9d6478900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" is_pre_election: true
I20260619 02:44:32.258930 25422 raft_consensus.cc:2454] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 536447d245f8461fa0694da9d6478900 in term 0.
I20260619 02:44:32.258927 25553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92730a4f85d4bc7affc5ceafc71f459" candidate_uuid: "536447d245f8461fa0694da9d6478900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9001467de444bbe8488846659e9dad0" is_pre_election: true
I20260619 02:44:32.259047 25553 raft_consensus.cc:2454] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 536447d245f8461fa0694da9d6478900 in term 0.
I20260619 02:44:32.259155 25617 leader_election.cc:302] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 536447d245f8461fa0694da9d6478900, bb64d40c8c174d06a54dab8f2bc8ec8f; no voters:
I20260619 02:44:32.259338 25751 raft_consensus.cc:2781] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:32.259378 25751 raft_consensus.cc:482] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:32.259393 25751 raft_consensus.cc:3037] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:32.260057 25751 raft_consensus.cc:504] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.260198 25751 leader_election.cc:288] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [CANDIDATE]: Term 1 election: Requested vote from peers bb64d40c8c174d06a54dab8f2bc8ec8f (127.7.137.1:42727), e9001467de444bbe8488846659e9dad0 (127.7.137.2:37183)
I20260619 02:44:32.260334 25422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92730a4f85d4bc7affc5ceafc71f459" candidate_uuid: "536447d245f8461fa0694da9d6478900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f"
I20260619 02:44:32.260354 25553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92730a4f85d4bc7affc5ceafc71f459" candidate_uuid: "536447d245f8461fa0694da9d6478900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9001467de444bbe8488846659e9dad0"
I20260619 02:44:32.260411 25422 raft_consensus.cc:3037] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:32.260424 25553 raft_consensus.cc:3037] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:32.261368 25422 raft_consensus.cc:2454] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 536447d245f8461fa0694da9d6478900 in term 1.
I20260619 02:44:32.261368 25553 raft_consensus.cc:2454] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 536447d245f8461fa0694da9d6478900 in term 1.
I20260619 02:44:32.261582 25618 leader_election.cc:302] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 536447d245f8461fa0694da9d6478900, e9001467de444bbe8488846659e9dad0; no voters:
I20260619 02:44:32.261719 25751 raft_consensus.cc:2781] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:32.261878 25751 raft_consensus.cc:686] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 1 LEADER]: Becoming Leader. State: Replica: 536447d245f8461fa0694da9d6478900, State: Running, Role: LEADER
I20260619 02:44:32.261991 25751 consensus_queue.cc:237] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } }
I20260619 02:44:32.262837 25280 catalog_manager.cc:5495] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 536447d245f8461fa0694da9d6478900 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "536447d245f8461fa0694da9d6478900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "536447d245f8461fa0694da9d6478900" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 44337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20260619 02:44:32.270926 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.268556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.7.137.3:44337)
0619 02:44:32.268616 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:32.268618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:32.268642 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:44:32.268743 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:32.268744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:32.268762 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:32.268844 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:32.268947 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.269406 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.270166 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:32.270357 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:32.270429 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:32.270589 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:32.270690 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:32.270780 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:32.270799 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 02:44:32.270819 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 02:44:32.272537 25553 raft_consensus.cc:1261] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 1 FOLLOWER]: Refusing update from remote peer 536447d245f8461fa0694da9d6478900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:32.272594 25422 raft_consensus.cc:1261] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 1 FOLLOWER]: Refusing update from remote peer 536447d245f8461fa0694da9d6478900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:32.272898 25754 consensus_queue.cc:1035] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9001467de444bbe8488846659e9dad0" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:32.273020 25751 consensus_queue.cc:1035] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb64d40c8c174d06a54dab8f2bc8ec8f" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:32.276049 25758 mvcc.cc:205] Tried to move back new op lower bound from 7298404648026091520 to 7298404647985758208. Current Snapshot: MvccSnapshot[applied={T|T < 7298404648026091520}]
I20260619 02:44:32.276153 25759 mvcc.cc:205] Tried to move back new op lower bound from 7298404648026091520 to 7298404647985758208. Current Snapshot: MvccSnapshot[applied={T|T < 7298404648026091520 or (T in {7298404648026091520})}]
I20260619 02:44:32.277796 25279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:32.277896 25279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:32.278607 25279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4279a0aec1ce4d168608304f54917be2: Sending DeleteTablet for 3 replicas of tablet a92730a4f85d4bc7affc5ceafc71f459
I20260619 02:44:32.278908 25402 tablet_service.cc:1514] Processing DeleteTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:32 UTC) from {username='slave'} at 127.0.0.1:34654
I20260619 02:44:32.279064 25664 tablet_service.cc:1514] Processing DeleteTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:32 UTC) from {username='slave'} at 127.0.0.1:41952
I20260619 02:44:32.279116 25533 tablet_service.cc:1514] Processing DeleteTablet for tablet a92730a4f85d4bc7affc5ceafc71f459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:32 UTC) from {username='slave'} at 127.0.0.1:58302
I20260619 02:44:32.279311 25766 tablet_replica.cc:330] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f: stopping tablet replica
I20260619 02:44:32.279313 25768 tablet_replica.cc:330] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: stopping tablet replica
I20260619 02:44:32.279404 25766 raft_consensus.cc:2229] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:32.279505 25766 raft_consensus.cc:2258] T a92730a4f85d4bc7affc5ceafc71f459 P bb64d40c8c174d06a54dab8f2bc8ec8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:32.279543 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25340
I20260619 02:44:32.279531 25768 raft_consensus.cc:2229] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:32.279747 25768 raft_consensus.cc:2258] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:32.280521 25768 ts_tablet_manager.cc:1905] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:32.280652 25767 tablet_replica.cc:330] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: stopping tablet replica
I20260619 02:44:32.280891 25767 raft_consensus.cc:2229] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:44:32.280997 25767 raft_consensus.cc:2258] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:32.281424 25767 ts_tablet_manager.cc:1905] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:32.282496 25768 ts_tablet_manager.cc:1918] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:32.282580 25768 log.cc:1196] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/a92730a4f85d4bc7affc5ceafc71f459
I20260619 02:44:32.282821 25768 ts_tablet_manager.cc:1939] T a92730a4f85d4bc7affc5ceafc71f459 P e9001467de444bbe8488846659e9dad0: Deleting consensus metadata
I20260619 02:44:32.283224 25265 catalog_manager.cc:4841] TS e9001467de444bbe8488846659e9dad0 (127.7.137.2:37183): tablet a92730a4f85d4bc7affc5ceafc71f459 (table test-table [id=6af2579f43884dbe88a65c777b6b4912]) successfully deleted
I20260619 02:44:32.283327 25767 ts_tablet_manager.cc:1918] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:32.283399 25767 log.cc:1196] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/a92730a4f85d4bc7affc5ceafc71f459
I20260619 02:44:32.283643 25767 ts_tablet_manager.cc:1939] T a92730a4f85d4bc7affc5ceafc71f459 P 536447d245f8461fa0694da9d6478900: Deleting consensus metadata
I20260619 02:44:32.284024 25266 catalog_manager.cc:4841] TS 536447d245f8461fa0694da9d6478900 (127.7.137.3:44337): tablet a92730a4f85d4bc7affc5ceafc71f459 (table test-table [id=6af2579f43884dbe88a65c777b6b4912]) successfully deleted
W20260619 02:44:32.284871 25264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:42727 (error 108)
I20260619 02:44:32.285750 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25471
I20260619 02:44:32.286403 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:32.285866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:42727 (local address 127.0.0.1:34658)
0619 02:44:32.286323 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:32.286364 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:42727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439}
W20260619 02:44:32.290639 25264 catalog_manager.cc:4584] TS bb64d40c8c174d06a54dab8f2bc8ec8f (127.7.137.1:42727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a92730a4f85d4bc7affc5ceafc71f459: Network error: Client connection negotiation failed: client connection to 127.7.137.1:42727: connect: Connection refused (error 111)
I20260619 02:44:32.290757 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25602
I20260619 02:44:32.295892 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25249
2026-06-19T02:44:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (810 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:32 dist-test-slave-pdnb krb5kdc[25772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:32 dist-test-slave-pdnb krb5kdc[25772](info): set up 2 sockets
Jun 19 02:44:32 dist-test-slave-pdnb krb5kdc[25772](info): commencing operation
krb5kdc: starting...
W20260619 02:44:33.396524 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:33.507500 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:40297
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:40297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:33.586900 25785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:33.587085 25785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:33.587106 25785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:33.588212 25785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:33.588251 25785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:33.588265 25785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:33.588279 25785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:33.588290 25785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:33.589718 25785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:40297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:40297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:33.589941 25785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:33.590161 25785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:33.594951 25785 server_base.cc:1034] running on GCE node
W20260619 02:44:33.594977 25793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:33.594980 25791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:33.594980 25790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:33.595489 25785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:33.595762 25785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:33.596913 25785 hybrid_clock.cc:648] HybridClock initialized: now 1781837073596902 us; error 27 us; skew 500 ppm
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:33.599251 25785 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:33.599649 25785 webserver.cc:460] Webserver started at http://127.7.137.62:43423/ using document root <none> and password file <none>
I20260619 02:44:33.599869 25785 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:33.599932 25785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:33.600078 25785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:33.601135 25785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "7b64d368d0dc40b2f785424db463b503"
server_key_iv: "a7dfd9ffa110dea0a9a713dacfce318c"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.601501 25785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "7b64d368d0dc40b2f785424db463b503"
server_key_iv: "a7dfd9ffa110dea0a9a713dacfce318c"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.602734 25785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:33.603492 25800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.603653 25785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:33.603750 25785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "7b64d368d0dc40b2f785424db463b503"
server_key_iv: "a7dfd9ffa110dea0a9a713dacfce318c"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.603829 25785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:33.620895 25785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:33.622813 25785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:33.622977 25785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:33.626806 25785 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:40297
I20260619 02:44:33.626942 25852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:40297 every 8 connection(s)
I20260619 02:44:33.627228 25785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:33.628059 25853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:33.630400 25853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: Bootstrap starting.
I20260619 02:44:33.630904 25853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:33.631142 25853 log.cc:824] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:33.631844 25853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: No bootstrap required, opened a new log
I20260619 02:44:33.631932 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25785
I20260619 02:44:33.632035 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:33.632228 7716 external_mini_cluster.cc:1428] Setting key 514ef942faf66a98ddaf68679e499f29
I20260619 02:44:33.633855 25853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } }
I20260619 02:44:33.634042 25853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:33.634094 25853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6589f1ba3a6d44aaa0ee2c340d7428e1, State: Initialized, Role: FOLLOWER
I20260619 02:44:33.634227 25853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } }
I20260619 02:44:33.634315 25853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:33.634348 25853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:33.634402 25853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:33.635372 25853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } }
I20260619 02:44:33.635463 25853 leader_election.cc:302] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6589f1ba3a6d44aaa0ee2c340d7428e1; no voters:
I20260619 02:44:33.635648 25853 leader_election.cc:288] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:33.635666 25858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:33.635818 25858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [term 1 LEADER]: Becoming Leader. State: Replica: 6589f1ba3a6d44aaa0ee2c340d7428e1, State: Running, Role: LEADER
I20260619 02:44:33.635898 25858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } }
I20260619 02:44:33.635962 25853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:33.636205 25859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } } }
I20260619 02:44:33.636350 25859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:33.636191 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6589f1ba3a6d44aaa0ee2c340d7428e1. Latest consensus state: current_term: 1 leader_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6589f1ba3a6d44aaa0ee2c340d7428e1" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 40297 } } }
I20260619 02:44:33.636461 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:33.636862 25865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:33.637377 25865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:33.639159 25865 catalog_manager.cc:1347] Generated new cluster ID: aa8d0796337148b8a5291e244c68fb76
I20260619 02:44:33.639206 25865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:33.644570 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:33.632738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.7.137.62:40297)
0619 02:44:33.632904 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:33.632919 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:33.632942 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:44:33.633839 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:33.633845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:33.633888 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:33.633995 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:33.634841 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.635923 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.637087 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.637697 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.637827 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:33.640678 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:33.640682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:33.640682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:33.640693 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:33.642619 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.643036 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.643039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.643039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.643160 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.643373 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.643373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.643374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.643644 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:33.644193 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:33.644257 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 02:44:33.644381 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260619 02:44:33.649660 25865 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:33.650130 25865 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:33.656463 25865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: Generated new TSK 0
I20260619 02:44:33.656638 25865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:44:33.681289 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:33.760525 25879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:33.760749 25879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:33.760805 25879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:33.761941 25879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:33.761981 25879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:33.762006 25879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:33.763430 25879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:33.763674 25879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:33.763896 25879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:33.768754 25884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:33.768771 25885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:33.768791 25879 server_base.cc:1034] running on GCE node
W20260619 02:44:33.768765 25887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:33.769173 25879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:33.769421 25879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:33.770582 25879 hybrid_clock.cc:648] HybridClock initialized: now 1781837073770572 us; error 29 us; skew 500 ppm
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:33.773118 25879 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:44:33.773561 25879 webserver.cc:460] Webserver started at http://127.7.137.1:46723/ using document root <none> and password file <none>
I20260619 02:44:33.773793 25879 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:33.773861 25879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:33.773995 25879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:33.774996 25879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "a8da82c293694c39b84178a306d58223"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "408b5f92f4deed169e9e4d9070fc5e74"
server_key_iv: "9882fd10abeb8e06be7f53b4027b04d5"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.775375 25879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "a8da82c293694c39b84178a306d58223"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "408b5f92f4deed169e9e4d9070fc5e74"
server_key_iv: "9882fd10abeb8e06be7f53b4027b04d5"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.776690 25879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:33.777513 25894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.777683 25879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:33.777779 25879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "a8da82c293694c39b84178a306d58223"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "408b5f92f4deed169e9e4d9070fc5e74"
server_key_iv: "9882fd10abeb8e06be7f53b4027b04d5"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.777864 25879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:33.787675 25879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:33.789517 25879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:33.789670 25879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:33.789880 25879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:33.790175 25879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:33.790232 25879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.790272 25879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:33.790300 25879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.795989 25879 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:45547
I20260619 02:44:33.796030 26007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:45547 every 8 connection(s)
I20260619 02:44:33.796332 25879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:33.805563 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:33.797036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:48203 (local address 127.7.137.62:40297)
0619 02:44:33.797105 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:33.797106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:33.797306 (+ 200us) server_negotiation.cc:407] Connection header received
0619 02:44:33.797801 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:33.797801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:33.797809 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:33.797860 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:33.798834 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.799321 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.800387 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.800590 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.800653 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:33.802985 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:33.802986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:33.802986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:33.802992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:33.804260 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.804688 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.804689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.804689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.804784 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.805051 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.805052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.805052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.805150 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:33.805200 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:33.805435 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 02:44:33.805498 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:44:33.805727 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:33.796994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:40297 (local address 127.7.137.1:48203)
0619 02:44:33.797167 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:33.797211 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:33.797718 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:33.797932 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:33.797935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:33.798544 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:33.798744 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:33.798750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.799409 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.799410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:33.800314 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:33.800316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.800679 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.800680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:33.800857 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:33.801442 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:33.801453 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:33.802893 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:33.804349 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:33.804355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:33.804357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:33.804621 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:33.804869 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:33.804870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:33.804871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:33.804988 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:33.805207 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:33.805218 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:33.805326 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:44:33.805414 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:44:33.805530 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260619 02:44:33.805770 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 25879
I20260619 02:44:33.805853 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:33.806053 7716 external_mini_cluster.cc:1428] Setting key 6aa175b8def4c73cb4b467ba5ad6745e
I20260619 02:44:33.806260 26008 heartbeater.cc:347] Connected to a master server at 127.7.137.62:40297
I20260619 02:44:33.806348 26008 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:33.806517 26008 heartbeater.cc:511] Master 127.7.137.62:40297 requested a full tablet report, sending...
I20260619 02:44:33.806993 25817 ts_manager.cc:194] Registered new tserver with Master: a8da82c293694c39b84178a306d58223 (127.7.137.1:45547)
I20260619 02:44:33.807643 25817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:48203
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:44:33.829641 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:33.911031 26013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:33.911235 26013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:33.911271 26013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:33.912392 26013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:33.912457 26013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:33.912501 26013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:33.913949 26013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:33.914203 26013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:33.914417 26013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:33.919322 26021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:33.919334 26018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:33.919330 26019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:33.919476 26013 server_base.cc:1034] running on GCE node
I20260619 02:44:33.919705 26013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:33.919934 26013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:33.921113 26013 hybrid_clock.cc:648] HybridClock initialized: now 1781837073921086 us; error 38 us; skew 500 ppm
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:33.923813 26013 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:44:33.924239 26013 webserver.cc:460] Webserver started at http://127.7.137.2:42125/ using document root <none> and password file <none>
I20260619 02:44:33.924463 26013 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:33.924531 26013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:33.924681 26013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:33.925731 26013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "4c20a8d28d13437fade1764585d60808"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "68dd7202f97f8a3838e9686183276e65"
server_key_iv: "978e57fbe8154f4004619786a33b63f8"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.926087 26013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "4c20a8d28d13437fade1764585d60808"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "68dd7202f97f8a3838e9686183276e65"
server_key_iv: "978e57fbe8154f4004619786a33b63f8"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.927335 26013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:33.928009 26028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.928160 26013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:33.928253 26013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "4c20a8d28d13437fade1764585d60808"
format_stamp: "Formatted at 2026-06-19 02:44:33 on dist-test-slave-pdnb"
server_key: "68dd7202f97f8a3838e9686183276e65"
server_key_iv: "978e57fbe8154f4004619786a33b63f8"
server_key_version: "encryptionkey@0"
I20260619 02:44:33.928333 26013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:33.939042 26013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:33.941267 26013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:33.941438 26013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:33.941653 26013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:33.941938 26013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:33.941998 26013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.942035 26013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:33.942062 26013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:33.947819 26013 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:41777
I20260619 02:44:33.947865 26141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:41777 every 8 connection(s)
I20260619 02:44:33.948155 26013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:33.953788 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26013
I20260619 02:44:33.953910 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:33.954119 7716 external_mini_cluster.cc:1428] Setting key 42f75828d355a01212c3424ba90d444f
Jun 19 02:44:33 dist-test-slave-pdnb krb5kdc[25772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837073, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:33.957819 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:33.948849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:34191 (local address 127.7.137.62:40297)
0619 02:44:33.948921 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:33.948924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:33.949142 (+ 218us) server_negotiation.cc:407] Connection header received
0619 02:44:33.949672 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:33.949673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:33.949687 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:33.949740 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:33.950687 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.951192 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.952215 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.952436 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.952516 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:33.954803 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:33.954804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:33.954805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:33.954813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:33.956260 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.956788 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.956789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.956789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.956842 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:33.957149 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:33.957150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:33.957150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:33.957251 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:33.957299 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:33.957673 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 02:44:33.957746 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:44:33.958056 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:33.948804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:40297 (local address 127.7.137.2:34191)
0619 02:44:33.949010 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:33.949045 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:33.949561 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:33.949811 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:33.949815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:33.950390 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:33.950577 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:33.950581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.951293 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.951294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:33.952117 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:33.952119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:33.952522 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:33.952523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:33.952652 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:33.953134 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:33.953142 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:33.954683 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:33.956341 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:33.956348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:33.956351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:33.956672 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:33.956920 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:33.956920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:33.956921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:33.957068 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:33.957315 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:33.957321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:33.957493 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 02:44:33.957599 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:44:33.957773 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260619 02:44:33.958612 26142 heartbeater.cc:347] Connected to a master server at 127.7.137.62:40297
I20260619 02:44:33.958722 26142 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:33.958915 26142 heartbeater.cc:511] Master 127.7.137.62:40297 requested a full tablet report, sending...
I20260619 02:44:33.959262 25817 ts_manager.cc:194] Registered new tserver with Master: 4c20a8d28d13437fade1764585d60808 (127.7.137.2:41777)
I20260619 02:44:33.959654 25817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:34191
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:44:33.977268 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:34.058076 26147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:34.058272 26147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:34.058295 26147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:34.059434 26147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:34.059478 26147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:34.059504 26147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:34.060945 26147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:34.061175 26147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:34.061412 26147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:34.066293 26155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.066311 26153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.066308 26152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.066573 26147 server_base.cc:1034] running on GCE node
I20260619 02:44:34.066737 26147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:34.066918 26147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:34.068053 26147 hybrid_clock.cc:648] HybridClock initialized: now 1781837074068035 us; error 43 us; skew 500 ppm
Jun 19 02:44:34 dist-test-slave-pdnb krb5kdc[25772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:34.070593 26147 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:44:34.070994 26147 webserver.cc:460] Webserver started at http://127.7.137.3:39513/ using document root <none> and password file <none>
I20260619 02:44:34.071174 26147 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:34.071214 26147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:34.071321 26147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:34.072381 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "89e74fa58e9747409dbb9d1c911ae183"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0d39e34a0c860f346dc824b8e750a6a1"
server_key_iv: "31e3edb17a1671f88e1b20df5da014b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.072793 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "89e74fa58e9747409dbb9d1c911ae183"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0d39e34a0c860f346dc824b8e750a6a1"
server_key_iv: "31e3edb17a1671f88e1b20df5da014b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.074096 26147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:44:34.074882 26162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.075052 26147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:34.075138 26147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "89e74fa58e9747409dbb9d1c911ae183"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0d39e34a0c860f346dc824b8e750a6a1"
server_key_iv: "31e3edb17a1671f88e1b20df5da014b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.075191 26147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:34.101989 26147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:34.104038 26147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:34.104147 26147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:34.104331 26147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:34.104669 26147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:34.104703 26147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.104769 26147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:34.104803 26147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.110649 26147 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:38881
I20260619 02:44:34.110697 26275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:38881 every 8 connection(s)
I20260619 02:44:34.110997 26147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:34.112016 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26147
I20260619 02:44:34.112107 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:34.112305 7716 external_mini_cluster.cc:1428] Setting key 2713c96026ac251e47e20e92cd7a8c8b
Jun 19 02:44:34 dist-test-slave-pdnb krb5kdc[25772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:34.120566 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.111763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:52631 (local address 127.7.137.62:40297)
0619 02:44:34.111821 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.111839 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.111992 (+ 153us) server_negotiation.cc:407] Connection header received
0619 02:44:34.112620 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.112622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.112635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.112738 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.113709 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.114220 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.115211 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.115425 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.115511 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.117750 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.117752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.117752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:34.117759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.119182 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:34.119619 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:34.119621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:34.119621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:34.119680 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:34.119930 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:34.119931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:34.119931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:34.120049 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.120124 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.120394 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.120478 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:44:34.120774 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.111711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:40297 (local address 127.7.137.3:52631)
0619 02:44:34.111857 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.111898 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.112488 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.112819 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.112824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.113412 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:34.113606 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.113611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.114324 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.114325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.115120 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.115122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.115521 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.115522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.115648 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.116176 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:34.116184 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:34.117648 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:34.119283 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:34.119289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:34.119291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:34.119524 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:34.119749 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:34.119750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:34.119750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:34.119858 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:34.120115 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:34.120121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:34.120259 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.120360 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.120520 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260619 02:44:34.121254 26276 heartbeater.cc:347] Connected to a master server at 127.7.137.62:40297
I20260619 02:44:34.121330 26276 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:34.121521 26276 heartbeater.cc:511] Master 127.7.137.62:40297 requested a full tablet report, sending...
I20260619 02:44:34.121884 25817 ts_manager.cc:194] Registered new tserver with Master: 89e74fa58e9747409dbb9d1c911ae183 (127.7.137.3:38881)
I20260619 02:44:34.122354 25817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:52631
I20260619 02:44:34.124110 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:34.134531 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.125943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54508 (local address 127.7.137.62:40297)
0619 02:44:34.126008 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.126010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.128183 (+ 2173us) server_negotiation.cc:407] Connection header received
0619 02:44:34.128258 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.128259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.128269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.128321 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.129047 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.129516 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.130496 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.130766 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.130828 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.132179 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.132180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.132180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:34.132193 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.133639 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:34.133947 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:34.133947 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:34.133948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:34.133993 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:34.134143 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:34.134143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:34.134144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:34.134248 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.134323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.134415 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.134471 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:34.136163 25817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:34.138132 25817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:34.144183 26286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45547 (local address 127.0.0.1:39672)
0619 02:44:34.140338 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.140373 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.140444 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.140814 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.140817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.140819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.140886 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.140888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.142319 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.142320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.143165 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.143168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.143888 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.143968 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.143982 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.144054 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.144106 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260619 02:44:34.144371 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.7.137.2:41777)
0619 02:44:34.140520 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.140525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.140629 (+ 104us) server_negotiation.cc:407] Connection header received
0619 02:44:34.140710 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.140712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.140768 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.140846 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.141051 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.141770 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143146 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.143821 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143890 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.143945 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.144140 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.144185 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":97,"threads_started":1}
I20260619 02:44:34.144613 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.7.137.3:38881)
0619 02:44:34.140508 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.140512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.140857 (+ 345us) server_negotiation.cc:407] Connection header received
0619 02:44:34.140900 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.140902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.140917 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.140989 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.141058 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.142144 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143430 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.144109 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.144167 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.144213 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.144499 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.144536 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260619 02:44:34.144622 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.7.137.1:45547)
0619 02:44:34.140611 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.140615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.140625 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:34.140666 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.140669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.140684 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.140812 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.141160 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.142208 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143250 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.143803 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143889 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.143954 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.144068 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.144561 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260619 02:44:34.144819 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:38881 (local address 127.0.0.1:49396)
0619 02:44:34.140748 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.140757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.140800 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.140983 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.140983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.140985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.141014 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.141016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.142248 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.142249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.143350 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.143353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.144246 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.144247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.144317 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.144320 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.144366 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.144408 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":53,"threads_started":1}
I20260619 02:44:34.144829 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.140056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41777 (local address 127.0.0.1:50274)
0619 02:44:34.140556 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.140568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.140637 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.140932 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.140932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.140934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.140966 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.140968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.141865 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.141866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.143037 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.143040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.143979 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.143980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.144042 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.144046 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.144457 (+ 411us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.144496 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":50,"threads_started":1}
I20260619 02:44:34.145357 25942 tablet_service.cc:1467] Processing CreateTablet for tablet a181b1c10ac54120879d38f1a2de44fe (DEFAULT_TABLE table=test-table [id=50d0512e59304798a2030b837c466708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.145514 26076 tablet_service.cc:1467] Processing CreateTablet for tablet a181b1c10ac54120879d38f1a2de44fe (DEFAULT_TABLE table=test-table [id=50d0512e59304798a2030b837c466708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.145749 25942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181b1c10ac54120879d38f1a2de44fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.145874 26076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181b1c10ac54120879d38f1a2de44fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.148243 26290 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: Bootstrap starting.
I20260619 02:44:34.148772 26210 tablet_service.cc:1467] Processing CreateTablet for tablet a181b1c10ac54120879d38f1a2de44fe (DEFAULT_TABLE table=test-table [id=50d0512e59304798a2030b837c466708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.149013 26290 tablet_bootstrap.cc:654] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.149107 26210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181b1c10ac54120879d38f1a2de44fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.149601 26290 log.cc:824] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.150367 26290 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: No bootstrap required, opened a new log
I20260619 02:44:34.150449 26290 ts_tablet_manager.cc:1397] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:34.152335 26293 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Bootstrap starting.
I20260619 02:44:34.152498 26291 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Bootstrap starting.
I20260619 02:44:34.152506 26290 raft_consensus.cc:348] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.152679 26290 raft_consensus.cc:374] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.152745 26290 raft_consensus.cc:729] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8da82c293694c39b84178a306d58223, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.152887 26290 consensus_queue.cc:260] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.153100 26293 tablet_bootstrap.cc:654] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.153188 26291 tablet_bootstrap.cc:654] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.153177 26290 ts_tablet_manager.cc:1428] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 02:44:34.153318 26008 heartbeater.cc:503] Master 127.7.137.62:40297 was elected leader, sending a full tablet report...
I20260619 02:44:34.153694 26293 log.cc:824] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.153769 26291 log.cc:824] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.154368 26293 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: No bootstrap required, opened a new log
I20260619 02:44:34.154448 26293 ts_tablet_manager.cc:1397] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:44:34.155745 26293 raft_consensus.cc:348] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.155927 26293 raft_consensus.cc:374] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.155973 26293 raft_consensus.cc:729] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e74fa58e9747409dbb9d1c911ae183, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.156092 26293 consensus_queue.cc:260] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.156339 26293 ts_tablet_manager.cc:1428] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:44:34.156383 26276 heartbeater.cc:503] Master 127.7.137.62:40297 was elected leader, sending a full tablet report...
I20260619 02:44:34.156890 26291 tablet_bootstrap.cc:492] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: No bootstrap required, opened a new log
I20260619 02:44:34.156991 26291 ts_tablet_manager.cc:1397] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:44:34.158337 26291 raft_consensus.cc:348] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.158457 26291 raft_consensus.cc:374] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.158479 26291 raft_consensus.cc:729] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c20a8d28d13437fade1764585d60808, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.158597 26291 consensus_queue.cc:260] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.158854 26291 ts_tablet_manager.cc:1428] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:34.158896 26142 heartbeater.cc:503] Master 127.7.137.62:40297 was elected leader, sending a full tablet report...
I20260619 02:44:34.181795 26297 raft_consensus.cc:482] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:34.181895 26297 raft_consensus.cc:504] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.182288 26297 leader_election.cc:288] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8da82c293694c39b84178a306d58223 (127.7.137.1:45547), 4c20a8d28d13437fade1764585d60808 (127.7.137.2:41777)
I20260619 02:44:34.185272 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.182505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:36357 (local address 127.7.137.2:41777)
0619 02:44:34.182551 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.182552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.182563 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:34.182660 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.182661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.182675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.182740 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.182897 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.183411 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.184421 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.184899 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.184953 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.184962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.185162 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.185208 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:44:34.185273 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.182434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41777 (local address 127.7.137.3:36357)
0619 02:44:34.182496 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.182508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.182554 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.182758 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.182758 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.182761 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.182830 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.182832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.183506 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.183507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.184335 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.184338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.185002 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.185004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.185083 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.185087 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.185144 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.185187 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:44:34.185338 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.182539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:47503 (local address 127.7.137.1:45547)
0619 02:44:34.182580 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.182581 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.182738 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:44:34.182773 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.182774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.182786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.182846 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.183057 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.183627 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.184533 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.184921 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.185020 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.185031 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.185227 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.185268 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:34.185572 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.182447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45547 (local address 127.7.137.3:47503)
0619 02:44:34.182659 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.182670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.182730 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.182855 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.182855 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.182857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.182938 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.182939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.183712 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.183713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.184451 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.184453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.185030 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.185031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.185133 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.185139 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.185413 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.185468 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":54,"thread_start_us":103,"threads_started":1}
I20260619 02:44:34.185607 26096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181b1c10ac54120879d38f1a2de44fe" candidate_uuid: "89e74fa58e9747409dbb9d1c911ae183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c20a8d28d13437fade1764585d60808" is_pre_election: true
I20260619 02:44:34.185789 26096 raft_consensus.cc:2454] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89e74fa58e9747409dbb9d1c911ae183 in term 0.
I20260619 02:44:34.185925 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181b1c10ac54120879d38f1a2de44fe" candidate_uuid: "89e74fa58e9747409dbb9d1c911ae183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8da82c293694c39b84178a306d58223" is_pre_election: true
I20260619 02:44:34.186012 26164 leader_election.cc:302] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c20a8d28d13437fade1764585d60808, 89e74fa58e9747409dbb9d1c911ae183; no voters:
I20260619 02:44:34.186108 25962 raft_consensus.cc:2454] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89e74fa58e9747409dbb9d1c911ae183 in term 0.
I20260619 02:44:34.186169 26297 raft_consensus.cc:2781] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:34.186237 26297 raft_consensus.cc:482] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:34.186273 26297 raft_consensus.cc:3037] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:34.186940 26297 raft_consensus.cc:504] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.187083 26297 leader_election.cc:288] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [CANDIDATE]: Term 1 election: Requested vote from peers a8da82c293694c39b84178a306d58223 (127.7.137.1:45547), 4c20a8d28d13437fade1764585d60808 (127.7.137.2:41777)
I20260619 02:44:34.187222 26096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181b1c10ac54120879d38f1a2de44fe" candidate_uuid: "89e74fa58e9747409dbb9d1c911ae183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c20a8d28d13437fade1764585d60808"
I20260619 02:44:34.187260 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181b1c10ac54120879d38f1a2de44fe" candidate_uuid: "89e74fa58e9747409dbb9d1c911ae183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8da82c293694c39b84178a306d58223"
I20260619 02:44:34.187302 26096 raft_consensus.cc:3037] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:34.187320 25962 raft_consensus.cc:3037] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:34.188175 26096 raft_consensus.cc:2454] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89e74fa58e9747409dbb9d1c911ae183 in term 1.
I20260619 02:44:34.188175 25962 raft_consensus.cc:2454] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89e74fa58e9747409dbb9d1c911ae183 in term 1.
I20260619 02:44:34.188369 26164 leader_election.cc:302] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c20a8d28d13437fade1764585d60808, 89e74fa58e9747409dbb9d1c911ae183; no voters:
I20260619 02:44:34.188468 26297 raft_consensus.cc:2781] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:34.188611 26297 raft_consensus.cc:686] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 1 LEADER]: Becoming Leader. State: Replica: 89e74fa58e9747409dbb9d1c911ae183, State: Running, Role: LEADER
I20260619 02:44:34.188750 26297 consensus_queue.cc:237] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } }
I20260619 02:44:34.189520 25817 catalog_manager.cc:5495] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89e74fa58e9747409dbb9d1c911ae183 (127.7.137.3). New cstate: current_term: 1 leader_uuid: "89e74fa58e9747409dbb9d1c911ae183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89e74fa58e9747409dbb9d1c911ae183" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 } health_report { overall_health: UNKNOWN } } }
W20260619 02:44:34.198657 26143 tablet.cc:2369] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:34.200942 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.198875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.7.137.3:38881)
0619 02:44:34.198920 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.198922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.198940 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:44:34.198970 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.198971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.198985 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.199049 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:34.199127 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.199599 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.200272 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.200490 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.200546 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.200686 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:34.200786 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:34.200851 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.200876 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.200886 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:44:34.202550 25962 raft_consensus.cc:1261] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 1 FOLLOWER]: Refusing update from remote peer 89e74fa58e9747409dbb9d1c911ae183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:34.202558 26096 raft_consensus.cc:1261] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 1 FOLLOWER]: Refusing update from remote peer 89e74fa58e9747409dbb9d1c911ae183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:34.202857 26297 consensus_queue.cc:1035] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8da82c293694c39b84178a306d58223" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:34.203016 26300 consensus_queue.cc:1035] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c20a8d28d13437fade1764585d60808" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 41777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:34.205209 26303 mvcc.cc:205] Tried to move back new op lower bound from 7298404655931211776 to 7298404655877890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298404655931211776}]
I20260619 02:44:34.205933 26305 mvcc.cc:205] Tried to move back new op lower bound from 7298404655931211776 to 7298404655877890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298404655931211776}]
I20260619 02:44:34.207959 25817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:34.208032 25817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:34.208694 25817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6589f1ba3a6d44aaa0ee2c340d7428e1: Sending DeleteTablet for 3 replicas of tablet a181b1c10ac54120879d38f1a2de44fe
I20260619 02:44:34.209014 25942 tablet_service.cc:1514] Processing DeleteTablet for tablet a181b1c10ac54120879d38f1a2de44fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:34 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:39672
I20260619 02:44:34.209043 26210 tablet_service.cc:1514] Processing DeleteTablet for tablet a181b1c10ac54120879d38f1a2de44fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:34 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:49396
I20260619 02:44:34.209090 26076 tablet_service.cc:1514] Processing DeleteTablet for tablet a181b1c10ac54120879d38f1a2de44fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:34 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:50274
I20260619 02:44:34.209223 26313 tablet_replica.cc:330] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: stopping tablet replica
I20260619 02:44:34.209287 26314 tablet_replica.cc:330] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808: stopping tablet replica
I20260619 02:44:34.209309 26313 raft_consensus.cc:2229] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:44:34.209358 26314 raft_consensus.cc:2229] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:34.209414 26313 raft_consensus.cc:2258] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:34.209653 26312 tablet_replica.cc:330] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223: stopping tablet replica
I20260619 02:44:34.209673 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25879
I20260619 02:44:34.209807 26312 raft_consensus.cc:2229] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:34.209879 26313 ts_tablet_manager.cc:1905] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:34.210027 26312 raft_consensus.cc:2258] T a181b1c10ac54120879d38f1a2de44fe P a8da82c293694c39b84178a306d58223 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:34.212158 26313 ts_tablet_manager.cc:1918] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:34.212235 26313 log.cc:1196] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/a181b1c10ac54120879d38f1a2de44fe
I20260619 02:44:34.212543 26313 ts_tablet_manager.cc:1939] T a181b1c10ac54120879d38f1a2de44fe P 89e74fa58e9747409dbb9d1c911ae183: Deleting consensus metadata
I20260619 02:44:34.212945 25801 catalog_manager.cc:4841] TS 89e74fa58e9747409dbb9d1c911ae183 (127.7.137.3:38881): tablet a181b1c10ac54120879d38f1a2de44fe (table test-table [id=50d0512e59304798a2030b837c466708]) successfully deleted
I20260619 02:44:34.209437 26314 raft_consensus.cc:2258] T a181b1c10ac54120879d38f1a2de44fe P 4c20a8d28d13437fade1764585d60808 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:44:34.216708 25801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:45547 (error 108)
I20260619 02:44:34.216938 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26013
I20260619 02:44:34.217880 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.217223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:45547 (local address 127.0.0.1:39684)
0619 02:44:34.217794 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.217840 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546}
W20260619 02:44:34.218482 25801 catalog_manager.cc:4584] TS a8da82c293694c39b84178a306d58223 (127.7.137.1:45547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a181b1c10ac54120879d38f1a2de44fe: Network error: Client connection negotiation failed: client connection to 127.7.137.1:45547: connect: Connection refused (error 111)
I20260619 02:44:34.221546 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.221434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:41777 (local address 127.0.0.1:50290)
0619 02:44:34.221490 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.221519 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.2:41777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:44:34.221573 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26147
I20260619 02:44:34.226467 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 25785
2026-06-19T02:44:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1931 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T02:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:34Z Disabled control of system clock
I20260619 02:44:34.253811 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42823
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:42823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:34.332787 26319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:34.332984 26319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:34.333003 26319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:34.334066 26319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:34.334105 26319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:34.334118 26319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:34.334131 26319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:34.334142 26319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:34.335391 26319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:42823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:34.335569 26319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:34.335783 26319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:34.340422 26324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.340427 26325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.340430 26327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.340734 26319 server_base.cc:1034] running on GCE node
I20260619 02:44:34.341045 26319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:34.341318 26319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:34.342478 26319 hybrid_clock.cc:648] HybridClock initialized: now 1781837074342469 us; error 34 us; skew 500 ppm
I20260619 02:44:34.343691 26319 webserver.cc:460] Webserver started at http://127.7.137.62:35555/ using document root <none> and password file <none>
I20260619 02:44:34.343928 26319 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:34.343997 26319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:34.344129 26319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:34.345211 26319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "956d084ed2e04657958fef7f2989ed9b"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0b51b88cabdd8a40577eada7bececfd0"
server_key_iv: "e7ec2d60fe0e8c33a17e9c8d65d92eb5"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.345564 26319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "956d084ed2e04657958fef7f2989ed9b"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0b51b88cabdd8a40577eada7bececfd0"
server_key_iv: "e7ec2d60fe0e8c33a17e9c8d65d92eb5"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.346833 26319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:34.347664 26333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.347846 26319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:34.347949 26319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "956d084ed2e04657958fef7f2989ed9b"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "0b51b88cabdd8a40577eada7bececfd0"
server_key_iv: "e7ec2d60fe0e8c33a17e9c8d65d92eb5"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.348024 26319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:34.373550 26319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:34.373880 26319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:34.374039 26319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:34.378116 26319 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:42823
I20260619 02:44:34.378177 26385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:42823 every 8 connection(s)
I20260619 02:44:34.378548 26319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:34.379545 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.381994 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: Bootstrap starting.
I20260619 02:44:34.382601 26386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.382853 26386 log.cc:824] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.383553 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: No bootstrap required, opened a new log
I20260619 02:44:34.385373 26386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } }
I20260619 02:44:34.385527 26386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.385551 26386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d084ed2e04657958fef7f2989ed9b, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.385694 26386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } }
I20260619 02:44:34.385782 26386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:34.385816 26386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:34.385854 26386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:34.386339 26386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } }
I20260619 02:44:34.386432 26386 leader_election.cc:302] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 956d084ed2e04657958fef7f2989ed9b; no voters:
I20260619 02:44:34.386561 26386 leader_election.cc:288] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:34.386653 26389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:34.386796 26389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [term 1 LEADER]: Becoming Leader. State: Replica: 956d084ed2e04657958fef7f2989ed9b, State: Running, Role: LEADER
I20260619 02:44:34.386830 26386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:34.386901 26389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } }
I20260619 02:44:34.387270 26391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956d084ed2e04657958fef7f2989ed9b. Latest consensus state: current_term: 1 leader_uuid: "956d084ed2e04657958fef7f2989ed9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } } }
I20260619 02:44:34.387351 26390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "956d084ed2e04657958fef7f2989ed9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956d084ed2e04657958fef7f2989ed9b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 42823 } } }
I20260619 02:44:34.387363 26391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:34.387425 26390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:34.387600 26395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:34.388113 26395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:34.388640 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26319
I20260619 02:44:34.388796 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:34.388993 7716 external_mini_cluster.cc:1428] Setting key 217b92a681f7a06a7d54878d94e4e5fa
I20260619 02:44:34.389763 26395 catalog_manager.cc:1347] Generated new cluster ID: 9fa47b7fd4da465f8b88ae6a208112ab
I20260619 02:44:34.389817 26395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:34.395474 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.389437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.7.137.62:42823)
0619 02:44:34.389633 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.389641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.389939 (+ 298us) server_negotiation.cc:407] Connection header received
0619 02:44:34.390078 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.390086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.390160 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.390325 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.390494 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.392055 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.393130 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.393836 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.393968 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.394682 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.394686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.394686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:34.394928 (+ 242us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.394975 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:34.394992 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.395070 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.395150 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.395231 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260619 02:44:34.397763 26395 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:34.398201 26395 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:34.407153 26395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: Generated new TSK 0
I20260619 02:44:34.407332 26395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:34.414625 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--builtin_ntp_servers=127.7.137.20:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:34.495054 26410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:34.495278 26410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:34.495316 26410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:34.496446 26410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:34.496515 26410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:34.496559 26410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:34.497972 26410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:34.498219 26410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:34.498512 26410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:34.504776 26416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.504837 26410 server_base.cc:1034] running on GCE node
W20260619 02:44:34.504891 26418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.504781 26415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.505227 26410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:34.505476 26410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:34.506641 26410 hybrid_clock.cc:648] HybridClock initialized: now 1781837074506632 us; error 36 us; skew 500 ppm
I20260619 02:44:34.511672 26410 webserver.cc:460] Webserver started at http://127.7.137.1:43977/ using document root <none> and password file <none>
I20260619 02:44:34.512025 26410 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:34.512092 26410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:34.512230 26410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:34.513355 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "e3014c67a61041c0af9741a82e58d4cf"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "e9d4772505b252369a5eb009f03a0166"
server_key_iv: "fcd4a788d9df79acd1d4cd9499a04872"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.513710 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "e3014c67a61041c0af9741a82e58d4cf"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "e9d4772505b252369a5eb009f03a0166"
server_key_iv: "fcd4a788d9df79acd1d4cd9499a04872"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.515057 26410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:34.515806 26424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.516021 26410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:34.516121 26410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "e3014c67a61041c0af9741a82e58d4cf"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "e9d4772505b252369a5eb009f03a0166"
server_key_iv: "fcd4a788d9df79acd1d4cd9499a04872"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.516194 26410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:34.538287 26410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:34.538565 26410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:34.538672 26410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:34.538865 26410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:34.539156 26410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:34.539189 26410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.539212 26410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:34.539227 26410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.545192 26410 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:38831
I20260619 02:44:34.545245 26537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:38831 every 8 connection(s)
I20260619 02:44:34.545630 26410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:44:34.549891 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26410
I20260619 02:44:34.550017 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:34.550232 7716 external_mini_cluster.cc:1428] Setting key c3fe5d0f2f98781cb0749a23da102b4c
I20260619 02:44:34.551226 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.546419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:47977 (local address 127.7.137.62:42823)
0619 02:44:34.546481 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.546483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.546712 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:44:34.547263 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.547265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.547280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.547326 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.547997 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.548491 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.549529 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.549775 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.549852 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.550880 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.550881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.550882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:34.550887 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.550903 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:34.550907 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.550960 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.551086 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.551115 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:44:34.551246 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--builtin_ntp_servers=127.7.137.20:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:44:34.551322 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.546389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42823 (local address 127.7.137.1:47977)
0619 02:44:34.546565 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.546614 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.547142 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.547409 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.547413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.547421 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:34.547898 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.547902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.548586 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.548587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.549432 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.549434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.549916 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.549917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.550107 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.550717 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:34.550717 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:34.550751 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:34.550773 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:34.550980 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:34.550987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:34.550991 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.551071 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.551159 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260619 02:44:34.551875 26538 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42823
I20260619 02:44:34.551999 26538 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:34.552191 26538 heartbeater.cc:511] Master 127.7.137.62:42823 requested a full tablet report, sending...
I20260619 02:44:34.552803 26350 ts_manager.cc:194] Registered new tserver with Master: e3014c67a61041c0af9741a82e58d4cf (127.7.137.1:38831)
I20260619 02:44:34.553823 26350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.1:47977
W20260619 02:44:34.633498 26541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:34.633687 26541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:34.633708 26541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:34.634757 26541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:34.634789 26541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:34.634814 26541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:34.636044 26541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:34.636220 26541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:34.636538 26541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:34.641541 26547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.641541 26549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.641556 26546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.641775 26541 server_base.cc:1034] running on GCE node
I20260619 02:44:34.641963 26541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:34.642205 26541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:34.643363 26541 hybrid_clock.cc:648] HybridClock initialized: now 1781837074643349 us; error 27 us; skew 500 ppm
I20260619 02:44:34.644536 26541 webserver.cc:460] Webserver started at http://127.7.137.2:39867/ using document root <none> and password file <none>
I20260619 02:44:34.644846 26541 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:34.644932 26541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:34.645067 26541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:34.646103 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "c49d8da02f8b40dfac3dda3a9b192c5c"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "743e2edb65842ec4c40e8d5424b4179a"
server_key_iv: "d53b9c7ae175f55e43e9110b5449a9f7"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.646459 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "c49d8da02f8b40dfac3dda3a9b192c5c"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "743e2edb65842ec4c40e8d5424b4179a"
server_key_iv: "d53b9c7ae175f55e43e9110b5449a9f7"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.647756 26541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:34.648439 26555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.648619 26541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:34.648717 26541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "c49d8da02f8b40dfac3dda3a9b192c5c"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "743e2edb65842ec4c40e8d5424b4179a"
server_key_iv: "d53b9c7ae175f55e43e9110b5449a9f7"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.648810 26541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:34.668555 26541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:34.668928 26541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:34.669086 26541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:34.669332 26541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:34.669709 26541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:34.669754 26541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.669785 26541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:34.669808 26541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.675735 26541 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:39967
I20260619 02:44:34.675800 26668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:39967 every 8 connection(s)
I20260619 02:44:34.676108 26541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:34.677102 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26541
I20260619 02:44:34.677207 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:34.677404 7716 external_mini_cluster.cc:1428] Setting key 5e1404f14fae04eeee24a77e0e9e3db0
I20260619 02:44:34.678333 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--builtin_ntp_servers=127.7.137.20:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:44:34.681857 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.676875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:45065 (local address 127.7.137.62:42823)
0619 02:44:34.676920 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.676922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.677195 (+ 273us) server_negotiation.cc:407] Connection header received
0619 02:44:34.677873 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.677874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.677895 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.677945 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.678785 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.679281 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.680265 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.680477 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.680592 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.681572 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.681573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.681574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:34.681579 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.681596 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:34.681600 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.681670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.681751 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.681779 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:34.682101 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.676879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42823 (local address 127.7.137.2:45065)
0619 02:44:34.677060 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.677100 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.677763 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.678015 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.678020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.678038 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:34.678681 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.678686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.679373 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.679374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.680159 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.680161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.680616 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.680617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.680819 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.681420 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:34.681421 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:34.681460 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:34.681483 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:34.681670 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:34.681677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:34.681680 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.681753 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.681855 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260619 02:44:34.682579 26669 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42823
I20260619 02:44:34.682654 26669 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:34.682847 26669 heartbeater.cc:511] Master 127.7.137.62:42823 requested a full tablet report, sending...
I20260619 02:44:34.683243 26350 ts_manager.cc:194] Registered new tserver with Master: c49d8da02f8b40dfac3dda3a9b192c5c (127.7.137.2:39967)
I20260619 02:44:34.683773 26350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.2:45065
W20260619 02:44:34.760923 26672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:34.761108 26672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:34.761129 26672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:34.762187 26672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:34.762218 26672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:34.762243 26672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:34.763474 26672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:34.763650 26672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:34.763877 26672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:34.768872 26677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.768867 26680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:34.768867 26678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:34.768884 26672 server_base.cc:1034] running on GCE node
I20260619 02:44:34.769263 26672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:34.769441 26672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:34.770570 26672 hybrid_clock.cc:648] HybridClock initialized: now 1781837074770565 us; error 31 us; skew 500 ppm
I20260619 02:44:34.771731 26672 webserver.cc:460] Webserver started at http://127.7.137.3:37519/ using document root <none> and password file <none>
I20260619 02:44:34.771927 26672 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:34.771967 26672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:34.772053 26672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:34.773132 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "707d2389004e48a59dc4a72d10cf3afa"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "7368d1b45fbeb263f539e30c7c36b520"
server_key_iv: "9b8bc7aabb2261e82948e74c86aadfb6"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.773424 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "707d2389004e48a59dc4a72d10cf3afa"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "7368d1b45fbeb263f539e30c7c36b520"
server_key_iv: "9b8bc7aabb2261e82948e74c86aadfb6"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.774645 26672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:34.775380 26686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.775554 26672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:34.775656 26672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "707d2389004e48a59dc4a72d10cf3afa"
format_stamp: "Formatted at 2026-06-19 02:44:34 on dist-test-slave-pdnb"
server_key: "7368d1b45fbeb263f539e30c7c36b520"
server_key_iv: "9b8bc7aabb2261e82948e74c86aadfb6"
server_key_version: "encryptionkey@0"
I20260619 02:44:34.775734 26672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:34.798528 26672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:34.798823 26672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:34.798974 26672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:34.799201 26672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:34.799544 26672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:34.799618 26672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.799655 26672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:34.799683 26672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:34.805423 26672 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:43541
I20260619 02:44:34.805469 26799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:43541 every 8 connection(s)
I20260619 02:44:34.805780 26672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:34.810775 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.806463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:33887 (local address 127.7.137.62:42823)
0619 02:44:34.806547 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.806549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.806789 (+ 240us) server_negotiation.cc:407] Connection header received
0619 02:44:34.807236 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.807237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.807248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.807303 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.807980 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.808516 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.809550 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.809748 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.809812 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.810546 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.810547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.810548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:34.810552 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.810566 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:34.810570 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.810615 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.810694 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.810719 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:44:34.810969 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.806452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:42823 (local address 127.7.137.3:33887)
0619 02:44:34.806664 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.806697 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.807149 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.807379 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.807383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.807391 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:34.807881 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.807885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.808602 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.808603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.809433 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.809435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.809826 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.809827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.809930 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.810414 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:34.810414 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:34.810440 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:34.810460 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:34.810625 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:34.810628 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:34.810630 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.810680 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.810751 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
I20260619 02:44:34.811468 26800 heartbeater.cc:347] Connected to a master server at 127.7.137.62:42823
I20260619 02:44:34.811553 26800 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:34.811736 26800 heartbeater.cc:511] Master 127.7.137.62:42823 requested a full tablet report, sending...
I20260619 02:44:34.812140 26350 ts_manager.cc:194] Registered new tserver with Master: 707d2389004e48a59dc4a72d10cf3afa (127.7.137.3:43541)
I20260619 02:44:34.812548 26350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.3:33887
I20260619 02:44:34.813149 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26672
I20260619 02:44:34.813254 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:34.813429 7716 external_mini_cluster.cc:1428] Setting key 5942fb9e75949849df13c926561c9f0a
I20260619 02:44:34.814240 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:34.818178 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.815518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.7.137.62:42823)
0619 02:44:34.815594 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.815596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.815660 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:44:34.815699 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.815700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.815709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.815752 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:34.815891 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.816334 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.817105 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.817290 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.817348 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.817916 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:34.817917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:34.817917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:34.817921 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:34.817933 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:34.817937 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:34.818004 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.818067 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.818076 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:44:34.819628 26350 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:34.821425 26350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:34.827652 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.823611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38831 (local address 127.0.0.1:37392)
0619 02:44:34.823810 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.823846 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.823921 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.824254 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.824256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.824258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.824302 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.824303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825382 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.825383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.826535 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.826537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827389 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.827440 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.827448 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.827530 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.827580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":63,"threads_started":1}
I20260619 02:44:34.827663 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.823695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.7.137.1:38831)
0619 02:44:34.823866 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.823870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.823906 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:44:34.824090 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.824093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.824110 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.824165 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.824374 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825289 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.826635 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827247 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827282 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.827341 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.827546 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.827582 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260619 02:44:34.827852 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.823854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:39967 (local address 127.0.0.1:38284)
0619 02:44:34.824034 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.824043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.824096 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.824452 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.824453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.824455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.824482 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.824483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825818 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.825819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.826633 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.826635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827615 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.827684 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.827690 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.827736 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.827776 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260619 02:44:34.827956 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.823668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:43541 (local address 127.0.0.1:38416)
0619 02:44:34.823987 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:34.823999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:34.824050 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:34.824423 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:34.824424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:34.824426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:34.824471 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.824472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825793 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.825794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.826863 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:34.826866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827714 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:34.827769 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.827771 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:44:34.827799 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 02:44:34.827832 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":93,"threads_started":1}
I20260619 02:44:34.828153 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.824079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.7.137.2:39967)
0619 02:44:34.824223 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.824229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.824239 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:34.824279 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.824282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.824297 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.824392 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.824557 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825734 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.826721 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827417 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827487 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.827576 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.828036 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.828074 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260619 02:44:34.828152 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:34.823875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.7.137.3:43541)
0619 02:44:34.824266 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:34.824270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:34.824278 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:34.824310 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:34.824313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:34.824328 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:34.824543 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:34.824597 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.825699 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.826950 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:34.827627 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:34.827912 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:34.828010 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:34.828034 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:44:34.828071 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":79,"threads_started":1}
I20260619 02:44:34.828877 26603 tablet_service.cc:1467] Processing CreateTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d (DEFAULT_TABLE table=test-table [id=597855ff09654773b8b0e5479b5bf59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.829107 26734 tablet_service.cc:1467] Processing CreateTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d (DEFAULT_TABLE table=test-table [id=597855ff09654773b8b0e5479b5bf59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.829185 26472 tablet_service.cc:1467] Processing CreateTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d (DEFAULT_TABLE table=test-table [id=597855ff09654773b8b0e5479b5bf59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:34.829294 26603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d680ecda30c4db4a5bf76ac8581727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.829450 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d680ecda30c4db4a5bf76ac8581727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.829520 26472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d680ecda30c4db4a5bf76ac8581727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:34.832698 26816 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Bootstrap starting.
I20260619 02:44:34.832902 26815 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Bootstrap starting.
I20260619 02:44:34.832922 26814 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Bootstrap starting.
I20260619 02:44:34.833446 26816 tablet_bootstrap.cc:654] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.833467 26814 tablet_bootstrap.cc:654] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.833534 26815 tablet_bootstrap.cc:654] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:34.833765 26814 log.cc:824] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.833782 26816 log.cc:824] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.833829 26815 log.cc:824] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:34.834468 26814 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: No bootstrap required, opened a new log
I20260619 02:44:34.834491 26816 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: No bootstrap required, opened a new log
I20260619 02:44:34.834540 26814 ts_tablet_manager.cc:1397] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:34.834566 26816 ts_tablet_manager.cc:1397] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 02:44:34.834715 26815 tablet_bootstrap.cc:492] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: No bootstrap required, opened a new log
I20260619 02:44:34.834780 26815 ts_tablet_manager.cc:1397] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:34.836567 26816 raft_consensus.cc:348] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.836720 26816 raft_consensus.cc:374] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.836812 26816 raft_consensus.cc:729] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d2389004e48a59dc4a72d10cf3afa, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.836941 26816 consensus_queue.cc:260] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.837258 26816 ts_tablet_manager.cc:1428] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:44:34.837399 26800 heartbeater.cc:503] Master 127.7.137.62:42823 was elected leader, sending a full tablet report...
I20260619 02:44:34.837349 26814 raft_consensus.cc:348] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.837422 26815 raft_consensus.cc:348] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.837529 26814 raft_consensus.cc:374] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.837543 26815 raft_consensus.cc:374] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:34.837567 26814 raft_consensus.cc:729] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49d8da02f8b40dfac3dda3a9b192c5c, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.837574 26815 raft_consensus.cc:729] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3014c67a61041c0af9741a82e58d4cf, State: Initialized, Role: FOLLOWER
I20260619 02:44:34.837697 26815 consensus_queue.cc:260] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.837697 26814 consensus_queue.cc:260] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:34.837953 26814 ts_tablet_manager.cc:1428] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 02:44:34.837967 26815 ts_tablet_manager.cc:1428] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:34.838069 26538 heartbeater.cc:503] Master 127.7.137.62:42823 was elected leader, sending a full tablet report...
I20260619 02:44:34.838256 26669 heartbeater.cc:503] Master 127.7.137.62:42823 was elected leader, sending a full tablet report...
W20260619 02:44:34.926676 26670 tablet.cc:2369] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:44:35.046344 26539 tablet.cc:2369] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:44:35.056330 26801 tablet.cc:2369] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:35.090859 26820 raft_consensus.cc:482] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:35.090992 26820 raft_consensus.cc:504] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:35.091425 26820 leader_election.cc:288] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3014c67a61041c0af9741a82e58d4cf (127.7.137.1:38831), c49d8da02f8b40dfac3dda3a9b192c5c (127.7.137.2:39967)
I20260619 02:44:35.095341 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.091603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38831 (local address 127.7.137.3:40113)
0619 02:44:35.091660 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:35.091671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:35.091716 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:35.091918 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:35.091919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:35.091921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:35.092003 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:35.092005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.092995 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.092996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:35.094300 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:35.094303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.095044 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.095045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:35.095120 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:35.095131 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:44:35.095191 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:44:35.095245 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":53}
I20260619 02:44:35.095352 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.091782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:53715 (local address 127.7.137.2:39967)
0619 02:44:35.091824 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:35.091826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:35.091961 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:44:35.091999 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:35.091999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:35.092007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:35.092080 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:35.092194 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.092915 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.094405 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.094953 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.095029 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:35.095038 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:35.095237 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:44:35.095285 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:35.095347 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.091726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:39967 (local address 127.7.137.3:53715)
0619 02:44:35.091878 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:35.091887 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:35.091946 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:35.092083 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:35.092083 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:35.092084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:35.092134 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:35.092135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.092994 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.092996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:35.094300 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:35.094303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.095045 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.095046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:35.095135 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:35.095137 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 02:44:35.095191 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:44:35.095242 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":50,"thread_start_us":101,"threads_started":1}
I20260619 02:44:35.095352 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.091659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:40113 (local address 127.7.137.1:38831)
0619 02:44:35.091702 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:35.091704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:35.091715 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:35.091805 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:35.091807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:35.091821 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:35.091907 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:35.092081 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.092893 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.094405 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.094953 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.095029 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:35.095038 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:35.095237 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:44:35.095282 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:44:35.095714 26492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d680ecda30c4db4a5bf76ac8581727d" candidate_uuid: "707d2389004e48a59dc4a72d10cf3afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3014c67a61041c0af9741a82e58d4cf" is_pre_election: true
I20260619 02:44:35.095747 26623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d680ecda30c4db4a5bf76ac8581727d" candidate_uuid: "707d2389004e48a59dc4a72d10cf3afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" is_pre_election: true
I20260619 02:44:35.095875 26623 raft_consensus.cc:2454] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707d2389004e48a59dc4a72d10cf3afa in term 0.
I20260619 02:44:35.095875 26492 raft_consensus.cc:2454] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707d2389004e48a59dc4a72d10cf3afa in term 0.
I20260619 02:44:35.096112 26687 leader_election.cc:302] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707d2389004e48a59dc4a72d10cf3afa, c49d8da02f8b40dfac3dda3a9b192c5c; no voters:
I20260619 02:44:35.096283 26820 raft_consensus.cc:2781] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:35.096401 26820 raft_consensus.cc:482] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:35.096438 26820 raft_consensus.cc:3037] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:35.097163 26820 raft_consensus.cc:504] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:35.097313 26820 leader_election.cc:288] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [CANDIDATE]: Term 1 election: Requested vote from peers e3014c67a61041c0af9741a82e58d4cf (127.7.137.1:38831), c49d8da02f8b40dfac3dda3a9b192c5c (127.7.137.2:39967)
I20260619 02:44:35.097458 26492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d680ecda30c4db4a5bf76ac8581727d" candidate_uuid: "707d2389004e48a59dc4a72d10cf3afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3014c67a61041c0af9741a82e58d4cf"
I20260619 02:44:35.097465 26623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d680ecda30c4db4a5bf76ac8581727d" candidate_uuid: "707d2389004e48a59dc4a72d10cf3afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c"
I20260619 02:44:35.097539 26492 raft_consensus.cc:3037] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:35.097561 26623 raft_consensus.cc:3037] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:35.098189 26492 raft_consensus.cc:2454] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707d2389004e48a59dc4a72d10cf3afa in term 1.
I20260619 02:44:35.098305 26623 raft_consensus.cc:2454] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707d2389004e48a59dc4a72d10cf3afa in term 1.
I20260619 02:44:35.098378 26688 leader_election.cc:302] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707d2389004e48a59dc4a72d10cf3afa, e3014c67a61041c0af9741a82e58d4cf; no voters:
I20260619 02:44:35.098449 26820 raft_consensus.cc:2781] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:35.098584 26820 raft_consensus.cc:686] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 1 LEADER]: Becoming Leader. State: Replica: 707d2389004e48a59dc4a72d10cf3afa, State: Running, Role: LEADER
I20260619 02:44:35.098690 26820 consensus_queue.cc:237] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } }
I20260619 02:44:35.099540 26350 catalog_manager.cc:5495] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 707d2389004e48a59dc4a72d10cf3afa (127.7.137.3). New cstate: current_term: 1 leader_uuid: "707d2389004e48a59dc4a72d10cf3afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "707d2389004e48a59dc4a72d10cf3afa" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 43541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 } health_report { overall_health: UNKNOWN } } }
I20260619 02:44:35.109308 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.106888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.7.137.3:43541)
0619 02:44:35.106945 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:35.106947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:35.106956 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:44:35.107027 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:35.107028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:35.107048 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:35.107141 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:35.107277 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.107750 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.108526 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:35.108743 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:35.108818 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:35.108949 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:35.109045 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:35.109152 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:35.109202 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 02:44:35.109210 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:44:35.110895 26492 raft_consensus.cc:1261] T 8d680ecda30c4db4a5bf76ac8581727d P e3014c67a61041c0af9741a82e58d4cf [term 1 FOLLOWER]: Refusing update from remote peer 707d2389004e48a59dc4a72d10cf3afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:35.110895 26623 raft_consensus.cc:1261] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 1 FOLLOWER]: Refusing update from remote peer 707d2389004e48a59dc4a72d10cf3afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:35.111240 26820 consensus_queue.cc:1035] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c49d8da02f8b40dfac3dda3a9b192c5c" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:35.111380 26824 consensus_queue.cc:1035] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3014c67a61041c0af9741a82e58d4cf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:35.115864 26349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:35.115958 26349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:35.116693 26349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 956d084ed2e04657958fef7f2989ed9b: Sending DeleteTablet for 3 replicas of tablet 8d680ecda30c4db4a5bf76ac8581727d
I20260619 02:44:35.117023 26472 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:35 UTC) from {username='slave'} at 127.0.0.1:37392
I20260619 02:44:35.117053 26603 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:35 UTC) from {username='slave'} at 127.0.0.1:38284
I20260619 02:44:35.117172 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26410
I20260619 02:44:35.117225 26471 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:35 UTC) from {username='slave'} at 127.0.0.1:37392
I20260619 02:44:35.117331 26836 tablet_replica.cc:330] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: stopping tablet replica
I20260619 02:44:35.117434 26836 raft_consensus.cc:2229] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:35.117522 26836 raft_consensus.cc:2258] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:35.118036 26734 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d680ecda30c4db4a5bf76ac8581727d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:35 UTC) from {username='slave'} at 127.0.0.1:38416
I20260619 02:44:35.118250 26838 tablet_replica.cc:330] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: stopping tablet replica
I20260619 02:44:35.118351 26838 raft_consensus.cc:2229] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:44:35.118439 26838 raft_consensus.cc:2258] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:35.118803 26838 ts_tablet_manager.cc:1905] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:35.119416 26836 ts_tablet_manager.cc:1905] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:35.120440 26838 ts_tablet_manager.cc:1918] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:35.120518 26838 log.cc:1196] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/8d680ecda30c4db4a5bf76ac8581727d
I20260619 02:44:35.120801 26838 ts_tablet_manager.cc:1939] T 8d680ecda30c4db4a5bf76ac8581727d P 707d2389004e48a59dc4a72d10cf3afa: Deleting consensus metadata
I20260619 02:44:35.121263 26836 ts_tablet_manager.cc:1918] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:35.121387 26836 log.cc:1196] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/8d680ecda30c4db4a5bf76ac8581727d
I20260619 02:44:35.121652 26334 catalog_manager.cc:4841] TS 707d2389004e48a59dc4a72d10cf3afa (127.7.137.3:43541): tablet 8d680ecda30c4db4a5bf76ac8581727d (table test-table [id=597855ff09654773b8b0e5479b5bf59d]) successfully deleted
I20260619 02:44:35.121776 26836 ts_tablet_manager.cc:1939] T 8d680ecda30c4db4a5bf76ac8581727d P c49d8da02f8b40dfac3dda3a9b192c5c: Deleting consensus metadata
I20260619 02:44:35.122128 26334 catalog_manager.cc:4841] TS c49d8da02f8b40dfac3dda3a9b192c5c (127.7.137.2:39967): tablet 8d680ecda30c4db4a5bf76ac8581727d (table test-table [id=597855ff09654773b8b0e5479b5bf59d]) successfully deleted
W20260619 02:44:35.125330 26335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:38831 (error 108)
I20260619 02:44:35.125514 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26541
I20260619 02:44:35.130378 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26672
I20260619 02:44:35.130411 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:35.130283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:38831 (local address 127.0.0.1:37406)
0619 02:44:35.130338 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:35.130384 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:44:35.130545 26335 catalog_manager.cc:4584] TS e3014c67a61041c0af9741a82e58d4cf (127.7.137.1:38831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d680ecda30c4db4a5bf76ac8581727d: Network error: Client connection negotiation failed: client connection to 127.7.137.1:38831: connect: Connection refused (error 111)
I20260619 02:44:35.135244 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26319
2026-06-19T02:44:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (907 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:35 dist-test-slave-pdnb krb5kdc[26842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:35 dist-test-slave-pdnb krb5kdc[26842](info): set up 2 sockets
Jun 19 02:44:35 dist-test-slave-pdnb krb5kdc[26842](info): commencing operation
krb5kdc: starting...
W20260619 02:44:36.164479 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:36.279333 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36243
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:36.358767 26855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:36.359009 26855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:36.359045 26855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:36.360175 26855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:36.360245 26855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:36.360270 26855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:36.360299 26855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:36.360330 26855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:36.361815 26855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:36.362087 26855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:36.362308 26855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:36.367264 26855 server_base.cc:1034] running on GCE node
W20260619 02:44:36.367198 26860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.367199 26861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.367229 26863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:36.367693 26855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:36.368031 26855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:36.369189 26855 hybrid_clock.cc:648] HybridClock initialized: now 1781837076369173 us; error 27 us; skew 500 ppm
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:36.371688 26855 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:36.372123 26855 webserver.cc:460] Webserver started at http://127.7.137.62:36963/ using document root <none> and password file <none>
I20260619 02:44:36.372363 26855 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:36.372409 26855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:36.372541 26855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:36.373616 26855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "9904eed523594144bb3a8591aa972972"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "34065da58866a02c3c8954c051285ac3"
server_key_iv: "7c81556a50b92b4615f7bc0e6f1186f2"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.373965 26855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "9904eed523594144bb3a8591aa972972"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "34065da58866a02c3c8954c051285ac3"
server_key_iv: "7c81556a50b92b4615f7bc0e6f1186f2"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.375370 26855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:36.376171 26870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.376364 26855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:36.376499 26855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "9904eed523594144bb3a8591aa972972"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "34065da58866a02c3c8954c051285ac3"
server_key_iv: "7c81556a50b92b4615f7bc0e6f1186f2"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.376576 26855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:36.390683 26855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:36.393014 26855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:36.393169 26855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:36.397212 26855 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36243
I20260619 02:44:36.397243 26922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36243 every 8 connection(s)
I20260619 02:44:36.397598 26855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:36.398455 26923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:36.400660 26923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: Bootstrap starting.
I20260619 02:44:36.401245 26923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:36.401536 26923 log.cc:824] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:36.402194 26923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: No bootstrap required, opened a new log
I20260619 02:44:36.403788 26923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } }
I20260619 02:44:36.403924 26923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:36.403949 26923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9904eed523594144bb3a8591aa972972, State: Initialized, Role: FOLLOWER
I20260619 02:44:36.404098 26923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } }
I20260619 02:44:36.404160 26923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:36.404181 26923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:36.404245 26923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:36.404841 26923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } }
I20260619 02:44:36.404872 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26855
I20260619 02:44:36.404979 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:36.404963 26923 leader_election.cc:302] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9904eed523594144bb3a8591aa972972; no voters:
I20260619 02:44:36.405153 26923 leader_election.cc:288] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:36.405171 26926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:36.405215 7716 external_mini_cluster.cc:1428] Setting key 1e2c778fa24c8a0616a37eea7b0270e9
I20260619 02:44:36.405290 26926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [term 1 LEADER]: Becoming Leader. State: Replica: 9904eed523594144bb3a8591aa972972, State: Running, Role: LEADER
I20260619 02:44:36.405360 26926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } }
I20260619 02:44:36.405450 26923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:36.405655 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9904eed523594144bb3a8591aa972972. Latest consensus state: current_term: 1 leader_uuid: "9904eed523594144bb3a8591aa972972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } } }
I20260619 02:44:36.405711 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:36.405671 26927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9904eed523594144bb3a8591aa972972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9904eed523594144bb3a8591aa972972" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36243 } } }
I20260619 02:44:36.405753 26927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:36.406453 26931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:36.407183 26931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:36.408659 26931 catalog_manager.cc:1347] Generated new cluster ID: e451d2df0f45448485c7ac73caafd90c
I20260619 02:44:36.408708 26931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:36.417616 26931 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:36.418203 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.405864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.7.137.62:36243)
0619 02:44:36.406055 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.406065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.406418 (+ 353us) server_negotiation.cc:407] Connection header received
0619 02:44:36.406556 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.406564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.406628 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.406782 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:36.407578 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.408958 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.410072 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.410688 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.410780 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.413623 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:36.413628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:36.413628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:36.413640 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:36.416283 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.416692 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.416695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.416695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.416828 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.417059 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.417060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.417060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.417376 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:36.417536 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.417870 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.418002 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260619 02:44:36.418298 26931 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:36.421870 26931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: Generated new TSK 0
I20260619 02:44:36.422040 26931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.137.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
Entry for principal kudu/127.7.137.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab.
I20260619 02:44:36.455466 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:36.555104 26949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:36.555327 26949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:36.555363 26949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:36.556671 26949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:36.556792 26949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:36.556874 26949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:36.558409 26949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.1.keytab
--principal=kudu/127.7.137.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:36.558645 26949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:36.558897 26949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:36.564432 26957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.564471 26954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.564438 26955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:36.564777 26949 server_base.cc:1034] running on GCE node
I20260619 02:44:36.564981 26949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:36.565179 26949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:36.566377 26949 hybrid_clock.cc:648] HybridClock initialized: now 1781837076566358 us; error 30 us; skew 500 ppm
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:36.568979 26949 init.cc:377] Logged in from keytab as kudu/127.7.137.1@KRBTEST.COM (short username kudu)
I20260619 02:44:36.569404 26949 webserver.cc:460] Webserver started at http://127.7.137.1:38153/ using document root <none> and password file <none>
I20260619 02:44:36.569638 26949 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:36.569705 26949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:36.569861 26949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:36.570887 26949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "19f065689fb24492af7d08765892db8c"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "0a7db2382f6fe90f75aaf913fc65fa5a"
server_key_iv: "40d2ec139a65e9d01c0f1390b06fa2b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.571249 26949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "19f065689fb24492af7d08765892db8c"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "0a7db2382f6fe90f75aaf913fc65fa5a"
server_key_iv: "40d2ec139a65e9d01c0f1390b06fa2b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.572685 26949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:36.573500 26964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.573679 26949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:36.573779 26949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "19f065689fb24492af7d08765892db8c"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "0a7db2382f6fe90f75aaf913fc65fa5a"
server_key_iv: "40d2ec139a65e9d01c0f1390b06fa2b3"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.573875 26949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:36.583964 26949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:36.585835 26949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:36.585996 26949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:36.586211 26949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:36.586513 26949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:36.586566 26949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.586604 26949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:36.586632 26949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.592427 26949 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:43829
I20260619 02:44:36.592466 27077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:43829 every 8 connection(s)
I20260619 02:44:36.592820 26949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.1@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:36.600955 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 26949
I20260619 02:44:36.601053 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:36.601267 7716 external_mini_cluster.cc:1428] Setting key 205798120545c3255f80d339d64fd070
I20260619 02:44:36.601754 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.593502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:48593 (local address 127.7.137.62:36243)
0619 02:44:36.593581 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.593583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.593721 (+ 138us) server_negotiation.cc:407] Connection header received
0619 02:44:36.594180 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.594181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.594190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.594240 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:36.595092 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.595614 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.596581 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.596864 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.596922 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.598906 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:36.598908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:36.598908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:36.598914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:36.600093 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.600459 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.600460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.600460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.600517 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.600910 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.600911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.600912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.601072 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:36.601149 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.601532 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.601635 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:44:36.601876 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.593451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36243 (local address 127.7.137.1:48593)
0619 02:44:36.593618 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.593652 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.594101 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.594288 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.594290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.594829 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:36.595015 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.595019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.595711 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.595711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.596497 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.596499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.596935 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.596935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.597036 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.597467 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:36.597475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:36.598811 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:36.600162 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.600167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.600169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.600373 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.600606 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.600607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.600607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.600802 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.601183 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:36.601190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:36.601351 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.601477 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.601646 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260619 02:44:36.602411 27078 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36243
I20260619 02:44:36.602510 27078 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:36.602708 27078 heartbeater.cc:511] Master 127.7.137.62:36243 requested a full tablet report, sending...
I20260619 02:44:36.603261 26887 ts_manager.cc:194] Registered new tserver with Master: 19f065689fb24492af7d08765892db8c (127.7.137.1:43829)
I20260619 02:44:36.603926 26887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.1@KRBTEST.COM'} at 127.7.137.1:48593
WARNING: no policy specified for kudu/127.7.137.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
Entry for principal kudu/127.7.137.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab.
I20260619 02:44:36.626972 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:36.706985 27083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:36.707185 27083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:36.707207 27083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:36.708355 27083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:36.708400 27083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:36.708425 27083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:36.709945 27083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.2.keytab
--principal=kudu/127.7.137.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:36.710191 27083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:36.710421 27083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:36.715441 27088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.715467 27089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.715459 27091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:36.715675 27083 server_base.cc:1034] running on GCE node
I20260619 02:44:36.715920 27083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:36.716120 27083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:36.717264 27083 hybrid_clock.cc:648] HybridClock initialized: now 1781837076717252 us; error 30 us; skew 500 ppm
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:36.719743 27083 init.cc:377] Logged in from keytab as kudu/127.7.137.2@KRBTEST.COM (short username kudu)
I20260619 02:44:36.720225 27083 webserver.cc:460] Webserver started at http://127.7.137.2:40385/ using document root <none> and password file <none>
I20260619 02:44:36.720472 27083 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:36.720543 27083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:36.720696 27083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:36.721920 27083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "00abf06dacf84c28816cc0e92ce95c70"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4452dd7d0f016c500ace747defe2b092"
server_key_iv: "86058236024399fc33586fa2a156c252"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.722340 27083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "00abf06dacf84c28816cc0e92ce95c70"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4452dd7d0f016c500ace747defe2b092"
server_key_iv: "86058236024399fc33586fa2a156c252"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.723677 27083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:36.724370 27098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.724557 27083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:36.724664 27083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "00abf06dacf84c28816cc0e92ce95c70"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4452dd7d0f016c500ace747defe2b092"
server_key_iv: "86058236024399fc33586fa2a156c252"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.724771 27083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:36.748052 27083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:36.750948 27083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:36.751181 27083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:36.751483 27083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:36.751856 27083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:36.751892 27083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.751916 27083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:36.751953 27083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.757898 27083 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:33443
I20260619 02:44:36.757946 27211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:33443 every 8 connection(s)
I20260619 02:44:36.758247 27083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:36.761615 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27083
I20260619 02:44:36.761731 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:36.761961 7716 external_mini_cluster.cc:1428] Setting key 6e78f757252b467a20e45e57c5c89ab8
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.2@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:36.768872 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.758959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:35871 (local address 127.7.137.62:36243)
0619 02:44:36.759026 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.759027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.759205 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:44:36.759705 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.759706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.759715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.759780 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:36.760818 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.761354 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.762844 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.763038 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.763094 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.765656 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:36.765657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:36.765658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:36.765664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:36.767144 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.767664 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.767665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.767665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.767717 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.768111 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.768112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.768112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.768239 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:36.768308 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.768657 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.768803 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:44:36.769073 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.758933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36243 (local address 127.7.137.2:35871)
0619 02:44:36.759077 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.759118 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.759620 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.759835 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.759838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.760480 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:36.760672 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.760678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.761523 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.761524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.762764 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.762767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.763124 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.763125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.763272 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.763821 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:36.763830 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:36.765558 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:36.767231 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.767238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.767241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.767585 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.767785 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.767786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.767787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.767937 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.768321 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:36.768328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:36.768508 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.768633 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.768831 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260619 02:44:36.769620 27212 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36243
I20260619 02:44:36.769735 27212 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:36.769941 27212 heartbeater.cc:511] Master 127.7.137.62:36243 requested a full tablet report, sending...
I20260619 02:44:36.770373 26887 ts_manager.cc:194] Registered new tserver with Master: 00abf06dacf84c28816cc0e92ce95c70 (127.7.137.2:33443)
WARNING: no policy specified for kudu/127.7.137.3@KRBTEST.COM; defaulting to no policy
I20260619 02:44:36.770833 26887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.2@KRBTEST.COM'} at 127.7.137.2:35871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
Entry for principal kudu/127.7.137.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab.
I20260619 02:44:36.787269 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:36.867573 27217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:36.867769 27217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:36.867791 27217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:36.868893 27217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:36.868974 27217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:36.869024 27217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:36.870457 27217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.3.keytab
--principal=kudu/127.7.137.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:36.870625 27217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:36.870788 27217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:36.875770 27225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.875794 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:36.875808 27223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:36.875962 27217 server_base.cc:1034] running on GCE node
I20260619 02:44:36.876335 27217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:36.876582 27217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:36.877748 27217 hybrid_clock.cc:648] HybridClock initialized: now 1781837076877747 us; error 44 us; skew 500 ppm
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:36.880132 27217 init.cc:377] Logged in from keytab as kudu/127.7.137.3@KRBTEST.COM (short username kudu)
I20260619 02:44:36.880587 27217 webserver.cc:460] Webserver started at http://127.7.137.3:43599/ using document root <none> and password file <none>
I20260619 02:44:36.880877 27217 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:36.880929 27217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:36.881065 27217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:36.882138 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "67bbf4d4af6747aaa7b5d442250b818b"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4a2aac63ebcfc6412f0c9a8421b1b7f7"
server_key_iv: "9cb35d94c256582ec28301dd1875fb13"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.882483 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "67bbf4d4af6747aaa7b5d442250b818b"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4a2aac63ebcfc6412f0c9a8421b1b7f7"
server_key_iv: "9cb35d94c256582ec28301dd1875fb13"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.883752 27217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:36.884488 27232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.884652 27217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:36.884770 27217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "67bbf4d4af6747aaa7b5d442250b818b"
format_stamp: "Formatted at 2026-06-19 02:44:36 on dist-test-slave-pdnb"
server_key: "4a2aac63ebcfc6412f0c9a8421b1b7f7"
server_key_iv: "9cb35d94c256582ec28301dd1875fb13"
server_key_version: "encryptionkey@0"
I20260619 02:44:36.884842 27217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:36.906668 27217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:36.909116 27217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:36.909226 27217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:36.909407 27217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:36.909726 27217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:36.909762 27217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.909807 27217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:36.909834 27217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:36.915431 27217 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:36703
I20260619 02:44:36.915500 27345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:36703 every 8 connection(s)
I20260619 02:44:36.915820 27217 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
Jun 19 02:44:36 dist-test-slave-pdnb krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837076, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.3@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:36.921999 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27217
I20260619 02:44:36.922135 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:36.922302 7716 external_mini_cluster.cc:1428] Setting key 60008649c1e5ec6b0526b0ae0b9b9ddd
I20260619 02:44:36.925410 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.916547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:54493 (local address 127.7.137.62:36243)
0619 02:44:36.916624 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.916627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.916823 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:44:36.917427 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.917428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.917441 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.917508 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:36.918411 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.918917 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.919878 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.920059 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.920114 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.922479 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:36.922481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:36.922482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:36.922488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:36.923951 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.924402 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.924403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.924404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.924468 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.924744 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.924745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.924745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.924880 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:36.924956 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.925233 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.925310 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:44:36.925611 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.916515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36243 (local address 127.7.137.3:54493)
0619 02:44:36.916676 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.916710 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.917305 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.917584 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.917587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.918144 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:36.918325 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.918330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.918995 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.918996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.919791 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.919793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.920131 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.920132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.920224 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.920709 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:44:36.920757 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:36.922384 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:36.924034 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.924040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.924043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.924260 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.924536 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:44:36.924536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:44:36.924537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:44:36.924650 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:44:36.924959 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:36.924966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:36.925102 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.925212 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.925378 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260619 02:44:36.926103 27346 heartbeater.cc:347] Connected to a master server at 127.7.137.62:36243
I20260619 02:44:36.926178 27346 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:36.926383 27346 heartbeater.cc:511] Master 127.7.137.62:36243 requested a full tablet report, sending...
I20260619 02:44:36.926774 26887 ts_manager.cc:194] Registered new tserver with Master: 67bbf4d4af6747aaa7b5d442250b818b (127.7.137.3:36703)
I20260619 02:44:36.927167 26887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.137.3@KRBTEST.COM'} at 127.7.137.3:54493
I20260619 02:44:36.934520 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:36.941540 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.935881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52320 (local address 127.7.137.62:36243)
0619 02:44:36.935922 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.935924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.935995 (+ 71us) server_negotiation.cc:407] Connection header received
0619 02:44:36.936050 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.936050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.936058 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.936112 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:36.936632 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.937151 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.937960 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.938136 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.938189 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.939280 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:36.939281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:36.939281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:36.939287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:36.940571 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.940947 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.940948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.940949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.940999 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:36.941152 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:36.941152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:36.941153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:36.941274 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:36.941379 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.941425 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.941479 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:36.943041 26887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:36.944612 26887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:36.950982 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.946887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:36703 (local address 127.0.0.1:60382)
0619 02:44:36.947271 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.947282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.947341 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.947810 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.947811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.947813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:36.947904 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.947906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.949195 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.949196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.949934 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.949936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.950732 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.950733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.950795 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.950806 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.950884 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.950928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260619 02:44:36.951035 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.947053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.7.137.3:36703)
0619 02:44:36.947604 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.947608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.947618 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:44:36.947653 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.947655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.947668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.947736 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:36.947999 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.949096 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.950026 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.950635 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.950709 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.950762 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.950913 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.950952 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":88,"threads_started":1}
I20260619 02:44:36.951730 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.946883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:43829 (local address 127.0.0.1:42434)
0619 02:44:36.947148 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.947186 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.947273 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.947728 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.947731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.947733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:36.947795 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.947797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.949166 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.949167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.950247 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.950250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.951481 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.951482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.951553 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.951558 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.951623 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.951664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":42,"threads_started":1}
I20260619 02:44:36.951779 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.947100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.7.137.1:43829)
0619 02:44:36.947371 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.947387 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.947398 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:36.947439 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.947441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.947455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.947549 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:36.947880 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.949054 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.950349 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.951061 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.951135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.951195 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.951652 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.951696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":81,"threads_started":1}
I20260619 02:44:36.951814 27280 tablet_service.cc:1467] Processing CreateTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 (DEFAULT_TABLE table=test-table [id=faa9274540824df3970c3b84030a999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:36.952172 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.946914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33443 (local address 127.0.0.1:49662)
0619 02:44:36.947393 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:36.947401 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:36.947471 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:36.947862 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:36.947862 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:36.947864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:36.947896 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.947897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.948808 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.948809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.950258 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:36.950260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.951934 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.951935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:36.951996 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.952001 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:36.952053 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 02:44:36.952101 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":70,"threads_started":1}
I20260619 02:44:36.952206 27280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fd8753505d4c9fa8037fea9e372da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:36.952414 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:36.947167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.7.137.2:33443)
0619 02:44:36.947344 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:36.947349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:36.947465 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:44:36.947535 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:36.947538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:36.947554 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:36.947660 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:36.947980 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.948665 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.951234 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:36.951828 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:36.951935 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:36.951990 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:36.952268 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 02:44:36.952319 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260619 02:44:36.952740 27012 tablet_service.cc:1467] Processing CreateTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 (DEFAULT_TABLE table=test-table [id=faa9274540824df3970c3b84030a999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:36.953079 27012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fd8753505d4c9fa8037fea9e372da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:36.953086 27146 tablet_service.cc:1467] Processing CreateTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 (DEFAULT_TABLE table=test-table [id=faa9274540824df3970c3b84030a999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:36.953411 27146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fd8753505d4c9fa8037fea9e372da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:36.955772 27363 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Bootstrap starting.
I20260619 02:44:36.956082 27364 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Bootstrap starting.
I20260619 02:44:36.956322 27365 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Bootstrap starting.
I20260619 02:44:36.956657 27363 tablet_bootstrap.cc:654] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:36.956807 27364 tablet_bootstrap.cc:654] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:36.957006 27365 tablet_bootstrap.cc:654] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:36.957262 27363 log.cc:824] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:36.957389 27364 log.cc:824] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:36.957641 27365 log.cc:824] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:36.958092 27363 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: No bootstrap required, opened a new log
I20260619 02:44:36.958163 27363 ts_tablet_manager.cc:1397] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:36.958317 27365 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: No bootstrap required, opened a new log
I20260619 02:44:36.958379 27365 ts_tablet_manager.cc:1397] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:36.960173 27363 raft_consensus.cc:348] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.960223 27365 raft_consensus.cc:348] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.960361 27365 raft_consensus.cc:374] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:36.960361 27363 raft_consensus.cc:374] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:36.960397 27363 raft_consensus.cc:729] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bbf4d4af6747aaa7b5d442250b818b, State: Initialized, Role: FOLLOWER
I20260619 02:44:36.960397 27365 raft_consensus.cc:729] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f065689fb24492af7d08765892db8c, State: Initialized, Role: FOLLOWER
I20260619 02:44:36.960510 27365 consensus_queue.cc:260] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.960510 27363 consensus_queue.cc:260] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.960812 27363 ts_tablet_manager.cc:1428] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:36.960812 27365 ts_tablet_manager.cc:1428] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:44:36.960832 27346 heartbeater.cc:503] Master 127.7.137.62:36243 was elected leader, sending a full tablet report...
I20260619 02:44:36.960924 27078 heartbeater.cc:503] Master 127.7.137.62:36243 was elected leader, sending a full tablet report...
I20260619 02:44:36.961889 27364 tablet_bootstrap.cc:492] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: No bootstrap required, opened a new log
I20260619 02:44:36.961962 27364 ts_tablet_manager.cc:1397] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260619 02:44:36.963321 27364 raft_consensus.cc:348] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.963503 27364 raft_consensus.cc:374] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:36.963563 27364 raft_consensus.cc:729] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00abf06dacf84c28816cc0e92ce95c70, State: Initialized, Role: FOLLOWER
I20260619 02:44:36.963685 27364 consensus_queue.cc:260] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:36.963987 27212 heartbeater.cc:503] Master 127.7.137.62:36243 was elected leader, sending a full tablet report...
I20260619 02:44:36.963994 27364 ts_tablet_manager.cc:1428] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260619 02:44:37.008869 27213 tablet.cc:2369] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:37.073076 27370 raft_consensus.cc:482] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:37.073186 27370 raft_consensus.cc:504] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:37.073580 27370 leader_election.cc:288] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19f065689fb24492af7d08765892db8c (127.7.137.1:43829), 00abf06dacf84c28816cc0e92ce95c70 (127.7.137.2:33443)
I20260619 02:44:37.076480 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.073814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:43829 (local address 127.7.137.3:53185)
0619 02:44:37.073871 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.073884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.073929 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.074135 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.074136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.074138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.074213 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.074215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.074901 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.074903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.075689 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.075691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.076236 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.076237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.076317 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.076321 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.076375 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.076417 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260619 02:44:37.076511 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.073877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:53185 (local address 127.7.137.1:43829)
0619 02:44:37.073925 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.073927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.073939 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:44:37.074011 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.074012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.074025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.074104 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.074285 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.074813 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.075778 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.076168 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.076221 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.076229 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.076397 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.076434 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 02:44:37.076679 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.073814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:33443 (local address 127.7.137.3:51253)
0619 02:44:37.074060 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.074070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.074126 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.074273 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.074274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.074276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.074345 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.074346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.074990 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.074991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.075704 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.075706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.076396 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.076397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.076465 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.076469 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.076527 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.076566 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":54,"thread_start_us":124,"threads_started":1}
I20260619 02:44:37.076674 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.073923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:51253 (local address 127.7.137.2:33443)
0619 02:44:37.073968 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.073969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.074141 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:44:37.074174 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.074175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.074190 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.074259 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.074422 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.074913 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.075824 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.076285 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.076374 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.076385 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.076551 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.076597 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:44:37.076829 27032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fd8753505d4c9fa8037fea9e372da5" candidate_uuid: "67bbf4d4af6747aaa7b5d442250b818b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f065689fb24492af7d08765892db8c" is_pre_election: true
I20260619 02:44:37.076996 27032 raft_consensus.cc:2454] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bbf4d4af6747aaa7b5d442250b818b in term 0.
I20260619 02:44:37.077245 27235 leader_election.cc:302] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f065689fb24492af7d08765892db8c, 67bbf4d4af6747aaa7b5d442250b818b; no voters:
I20260619 02:44:37.077301 27166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fd8753505d4c9fa8037fea9e372da5" candidate_uuid: "67bbf4d4af6747aaa7b5d442250b818b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00abf06dacf84c28816cc0e92ce95c70" is_pre_election: true
I20260619 02:44:37.077399 27370 raft_consensus.cc:2781] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:37.077440 27370 raft_consensus.cc:482] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:37.077454 27166 raft_consensus.cc:2454] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bbf4d4af6747aaa7b5d442250b818b in term 0.
I20260619 02:44:37.077471 27370 raft_consensus.cc:3037] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.078197 27370 raft_consensus.cc:504] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:37.078339 27370 leader_election.cc:288] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [CANDIDATE]: Term 1 election: Requested vote from peers 19f065689fb24492af7d08765892db8c (127.7.137.1:43829), 00abf06dacf84c28816cc0e92ce95c70 (127.7.137.2:33443)
I20260619 02:44:37.078495 27166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fd8753505d4c9fa8037fea9e372da5" candidate_uuid: "67bbf4d4af6747aaa7b5d442250b818b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00abf06dacf84c28816cc0e92ce95c70"
I20260619 02:44:37.078495 27032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fd8753505d4c9fa8037fea9e372da5" candidate_uuid: "67bbf4d4af6747aaa7b5d442250b818b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f065689fb24492af7d08765892db8c"
I20260619 02:44:37.078577 27032 raft_consensus.cc:3037] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.078577 27166 raft_consensus.cc:3037] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.079443 27166 raft_consensus.cc:2454] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bbf4d4af6747aaa7b5d442250b818b in term 1.
I20260619 02:44:37.079443 27032 raft_consensus.cc:2454] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bbf4d4af6747aaa7b5d442250b818b in term 1.
I20260619 02:44:37.079684 27234 leader_election.cc:302] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00abf06dacf84c28816cc0e92ce95c70, 67bbf4d4af6747aaa7b5d442250b818b; no voters:
I20260619 02:44:37.079794 27370 raft_consensus.cc:2781] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:37.079965 27370 raft_consensus.cc:686] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 1 LEADER]: Becoming Leader. State: Replica: 67bbf4d4af6747aaa7b5d442250b818b, State: Running, Role: LEADER
I20260619 02:44:37.080101 27370 consensus_queue.cc:237] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } }
I20260619 02:44:37.081014 26887 catalog_manager.cc:5495] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b reported cstate change: term changed from 0 to 1, leader changed from <none> to 67bbf4d4af6747aaa7b5d442250b818b (127.7.137.3). New cstate: current_term: 1 leader_uuid: "67bbf4d4af6747aaa7b5d442250b818b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bbf4d4af6747aaa7b5d442250b818b" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 36703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20260619 02:44:37.093552 27079 tablet.cc:2369] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:37.097803 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.095285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.7.137.3:36703)
0619 02:44:37.095333 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.095335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.095342 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:37.095436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.095437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.095456 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.095514 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:37.095649 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.096097 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.096943 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.097136 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.097210 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.097484 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:37.097582 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:37.097679 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.097698 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.097720 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 02:44:37.099613 27032 raft_consensus.cc:1261] T e2fd8753505d4c9fa8037fea9e372da5 P 19f065689fb24492af7d08765892db8c [term 1 FOLLOWER]: Refusing update from remote peer 67bbf4d4af6747aaa7b5d442250b818b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:37.099612 27166 raft_consensus.cc:1261] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 1 FOLLOWER]: Refusing update from remote peer 67bbf4d4af6747aaa7b5d442250b818b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:37.099958 27373 consensus_queue.cc:1035] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00abf06dacf84c28816cc0e92ce95c70" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:37.100107 27370 consensus_queue.cc:1035] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f065689fb24492af7d08765892db8c" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:37.103518 27377 mvcc.cc:205] Tried to move back new op lower bound from 7298404667797102592 to 7298404667720876032. Current Snapshot: MvccSnapshot[applied={T|T < 7298404667797102592 or (T in {7298404667797102592})}]
I20260619 02:44:37.105046 26886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:37.105137 26886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:37.105836 26886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9904eed523594144bb3a8591aa972972: Sending DeleteTablet for 3 replicas of tablet e2fd8753505d4c9fa8037fea9e372da5
I20260619 02:44:37.106161 27012 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:42434
I20260619 02:44:37.106235 27280 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:60382
I20260619 02:44:37.106259 27146 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fd8753505d4c9fa8037fea9e372da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='kudu', principal='kudu/127.7.137.62@KRBTEST.COM'} at 127.0.0.1:49662
I20260619 02:44:37.106482 27388 tablet_replica.cc:330] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: stopping tablet replica
I20260619 02:44:37.106587 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26949
I20260619 02:44:37.107314 27388 raft_consensus.cc:2229] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:37.107447 27388 raft_consensus.cc:2258] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:37.107630 27387 tablet_replica.cc:330] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: stopping tablet replica
I20260619 02:44:37.107724 27387 raft_consensus.cc:2229] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:44:37.107873 27387 raft_consensus.cc:2258] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:37.108126 27388 ts_tablet_manager.cc:1905] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:37.108335 27387 ts_tablet_manager.cc:1905] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:37.110026 27387 ts_tablet_manager.cc:1918] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:37.110020 27388 ts_tablet_manager.cc:1918] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:37.110095 27387 log.cc:1196] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/e2fd8753505d4c9fa8037fea9e372da5
I20260619 02:44:37.110199 27388 log.cc:1196] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/e2fd8753505d4c9fa8037fea9e372da5
I20260619 02:44:37.110386 27387 ts_tablet_manager.cc:1939] T e2fd8753505d4c9fa8037fea9e372da5 P 67bbf4d4af6747aaa7b5d442250b818b: Deleting consensus metadata
I20260619 02:44:37.110559 27388 ts_tablet_manager.cc:1939] T e2fd8753505d4c9fa8037fea9e372da5 P 00abf06dacf84c28816cc0e92ce95c70: Deleting consensus metadata
I20260619 02:44:37.110782 26874 catalog_manager.cc:4841] TS 67bbf4d4af6747aaa7b5d442250b818b (127.7.137.3:36703): tablet e2fd8753505d4c9fa8037fea9e372da5 (table test-table [id=faa9274540824df3970c3b84030a999f]) successfully deleted
I20260619 02:44:37.111003 26872 catalog_manager.cc:4841] TS 00abf06dacf84c28816cc0e92ce95c70 (127.7.137.2:33443): tablet e2fd8753505d4c9fa8037fea9e372da5 (table test-table [id=faa9274540824df3970c3b84030a999f]) successfully deleted
W20260619 02:44:37.113010 26873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:43829 (error 108)
I20260619 02:44:37.113233 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27083
I20260619 02:44:37.114982 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.114825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:43829 (local address 127.0.0.1:42444)
0619 02:44:37.114902 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.114949 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 02:44:37.115123 26873 catalog_manager.cc:4584] TS 19f065689fb24492af7d08765892db8c (127.7.137.1:43829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2fd8753505d4c9fa8037fea9e372da5: Network error: Client connection negotiation failed: client connection to 127.7.137.1:43829: connect: Connection refused (error 111)
I20260619 02:44:37.119174 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27217
I20260619 02:44:37.123915 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 26855
2026-06-19T02:44:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1989 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T02:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:37Z Disabled control of system clock
I20260619 02:44:37.151459 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35749
--webserver_interface=127.7.137.62
--webserver_port=0
--builtin_ntp_servers=127.7.137.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:35749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:37.229733 27393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:37.229935 27393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:37.229955 27393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:37.231000 27393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:37.231030 27393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:37.231043 27393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:37.231055 27393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:37.231067 27393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:37.232295 27393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:35749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:37.232486 27393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:37.232703 27393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:37.237437 27398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.237452 27399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.237432 27401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:37.237802 27393 server_base.cc:1034] running on GCE node
I20260619 02:44:37.238013 27393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:37.238263 27393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:37.239423 27393 hybrid_clock.cc:648] HybridClock initialized: now 1781837077239414 us; error 34 us; skew 500 ppm
I20260619 02:44:37.240644 27393 webserver.cc:460] Webserver started at http://127.7.137.62:41697/ using document root <none> and password file <none>
I20260619 02:44:37.240970 27393 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:37.241052 27393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:37.241186 27393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:37.242266 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "cc50b781812c45cc92f4682201d937c6"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "315f8d6f3db225e0ea0bbec1c4ee176c"
server_key_iv: "7ff0bb882c348c3ccd009253b0fe558c"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.242616 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "cc50b781812c45cc92f4682201d937c6"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "315f8d6f3db225e0ea0bbec1c4ee176c"
server_key_iv: "7ff0bb882c348c3ccd009253b0fe558c"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.243881 27393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:37.244618 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.244822 27393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:37.244892 27393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "cc50b781812c45cc92f4682201d937c6"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "315f8d6f3db225e0ea0bbec1c4ee176c"
server_key_iv: "7ff0bb882c348c3ccd009253b0fe558c"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.244978 27393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:37.255198 27393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:37.255470 27393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:37.255611 27393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:37.259613 27393 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:35749
I20260619 02:44:37.259670 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:35749 every 8 connection(s)
I20260619 02:44:37.260007 27393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:37.261003 27460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:37.263212 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: Bootstrap starting.
I20260619 02:44:37.263804 27460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:37.264112 27460 log.cc:824] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:37.264822 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: No bootstrap required, opened a new log
I20260619 02:44:37.265717 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27393
I20260619 02:44:37.265846 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:37.266042 7716 external_mini_cluster.cc:1428] Setting key 1b75a74517980fcac02194ebeec43d46
I20260619 02:44:37.266741 27460 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } }
I20260619 02:44:37.266904 27460 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:37.266949 27460 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc50b781812c45cc92f4682201d937c6, State: Initialized, Role: FOLLOWER
I20260619 02:44:37.267091 27460 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } }
I20260619 02:44:37.267184 27460 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:37.267221 27460 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:37.267262 27460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.267765 27460 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } }
I20260619 02:44:37.267885 27460 leader_election.cc:302] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc50b781812c45cc92f4682201d937c6; no voters:
I20260619 02:44:37.268057 27460 leader_election.cc:288] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:37.268100 27465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:37.268231 27465 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [term 1 LEADER]: Becoming Leader. State: Replica: cc50b781812c45cc92f4682201d937c6, State: Running, Role: LEADER
I20260619 02:44:37.268308 27460 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:37.268311 27465 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } }
I20260619 02:44:37.268623 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc50b781812c45cc92f4682201d937c6. Latest consensus state: current_term: 1 leader_uuid: "cc50b781812c45cc92f4682201d937c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } } }
I20260619 02:44:37.268698 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:37.268632 27467 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc50b781812c45cc92f4682201d937c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc50b781812c45cc92f4682201d937c6" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35749 } } }
I20260619 02:44:37.268919 27467 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6 [sys.catalog]: This master's current role is: LEADER
W20260619 02:44:37.270005 27481 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:37.270062 27481 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:37.270116 27471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:37.270254 27471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:37.271682 27471 catalog_manager.cc:1347] Generated new cluster ID: ee688234b7fd4260aefc5870aa15c9c6
I20260619 02:44:37.271733 27471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:37.272182 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.266591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.7.137.62:35749)
0619 02:44:37.266774 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.266781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.266808 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:44:37.266899 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.266904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.266985 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.267153 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:37.267342 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.268983 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.270164 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.270659 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.270793 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.271486 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:37.271489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:37.271489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:37.271687 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:37.271726 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:37.271740 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:37.271799 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.271910 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.271989 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 02:44:37.292642 27471 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:37.293256 27471 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:37.298448 27471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: Generated new TSK 0
I20260619 02:44:37.298638 27471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:37.307392 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.1:0
--local_ip_for_outbound_sockets=127.7.137.1
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--builtin_ntp_servers=127.7.137.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:37.401829 27484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:37.402052 27484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:37.402077 27484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:37.403424 27484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:37.403468 27484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:37.403496 27484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.1
I20260619 02:44:37.405073 27484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.137.1
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.1
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:37.405323 27484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:37.405647 27484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:37.410804 27492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.410830 27489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.410822 27490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:37.410969 27484 server_base.cc:1034] running on GCE node
I20260619 02:44:37.411165 27484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:37.411401 27484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:37.412523 27484 hybrid_clock.cc:648] HybridClock initialized: now 1781837077412492 us; error 46 us; skew 500 ppm
I20260619 02:44:37.413751 27484 webserver.cc:460] Webserver started at http://127.7.137.1:39609/ using document root <none> and password file <none>
I20260619 02:44:37.413987 27484 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:37.414031 27484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:37.414171 27484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:37.415180 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data/instance:
uuid: "fe3a602937334054a4892d09408b73bf"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "9180e8fdacc6a867e64dce7565b29cd4"
server_key_iv: "db098750e2c11640b0f1edcc0179ead3"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.415555 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance:
uuid: "fe3a602937334054a4892d09408b73bf"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "9180e8fdacc6a867e64dce7565b29cd4"
server_key_iv: "db098750e2c11640b0f1edcc0179ead3"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.416850 27484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:37.417559 27498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.417713 27484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:37.417810 27484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal
uuid: "fe3a602937334054a4892d09408b73bf"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "9180e8fdacc6a867e64dce7565b29cd4"
server_key_iv: "db098750e2c11640b0f1edcc0179ead3"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.417889 27484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:37.433723 27484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:37.434029 27484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:37.434175 27484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:37.434404 27484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:37.434715 27484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:37.434767 27484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.434806 27484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:37.434831 27484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.440490 27484 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.1:33495
I20260619 02:44:37.440567 27611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.1:33495 every 8 connection(s)
I20260619 02:44:37.440925 27484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/data/info.pb
I20260619 02:44:37.442991 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27484
I20260619 02:44:37.443102 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-0/wal/instance
I20260619 02:44:37.443295 7716 external_mini_cluster.cc:1428] Setting key bbaac2d786ec824dcc67e45f4f98b6fe
I20260619 02:44:37.444351 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.2:0
--local_ip_for_outbound_sockets=127.7.137.2
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--builtin_ntp_servers=127.7.137.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:44:37.446710 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.441621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.1:53501 (local address 127.7.137.62:35749)
0619 02:44:37.441667 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.441669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.441889 (+ 220us) server_negotiation.cc:407] Connection header received
0619 02:44:37.442339 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.442340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.442354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.442417 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:37.443052 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.443784 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.445085 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.445471 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.445550 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.446485 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:37.446486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:37.446486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:37.446490 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:37.446504 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:37.446508 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:37.446552 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.446627 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.446653 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:37.446949 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.441580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:35749 (local address 127.7.137.1:53501)
0619 02:44:37.441759 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.441799 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.442262 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.442466 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.442469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.442477 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:37.442968 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.442972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.443902 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.443903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.444979 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.444983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.445586 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.445588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.445808 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.446336 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:37.446337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:37.446377 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:37.446406 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:37.446554 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:37.446558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:37.446561 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.446629 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.446743 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260619 02:44:37.447397 27612 heartbeater.cc:347] Connected to a master server at 127.7.137.62:35749
I20260619 02:44:37.447487 27612 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:37.447664 27612 heartbeater.cc:511] Master 127.7.137.62:35749 requested a full tablet report, sending...
I20260619 02:44:37.448196 27424 ts_manager.cc:194] Registered new tserver with Master: fe3a602937334054a4892d09408b73bf (127.7.137.1:33495)
I20260619 02:44:37.449239 27424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.1:53501
W20260619 02:44:37.524816 27615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:37.525010 27615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:37.525031 27615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:37.526096 27615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:37.526129 27615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:37.526154 27615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.2
I20260619 02:44:37.527386 27615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.137.2
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.2
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:37.527556 27615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:37.527779 27615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:37.532737 27615 server_base.cc:1034] running on GCE node
W20260619 02:44:37.532677 27621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.532685 27620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.532689 27623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:37.533121 27615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:37.533380 27615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:37.534543 27615 hybrid_clock.cc:648] HybridClock initialized: now 1781837077534533 us; error 29 us; skew 500 ppm
I20260619 02:44:37.535770 27615 webserver.cc:460] Webserver started at http://127.7.137.2:39227/ using document root <none> and password file <none>
I20260619 02:44:37.536008 27615 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:37.536080 27615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:37.536216 27615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:37.537302 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data/instance:
uuid: "b45722a1d30a4679a447e47480721233"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "b9009ded3f80dd6165062654df22855c"
server_key_iv: "45c91225e10a6959bf5287d935bbc11a"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.537678 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance:
uuid: "b45722a1d30a4679a447e47480721233"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "b9009ded3f80dd6165062654df22855c"
server_key_iv: "45c91225e10a6959bf5287d935bbc11a"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.539000 27615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:37.539765 27629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.539968 27615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:37.540073 27615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal
uuid: "b45722a1d30a4679a447e47480721233"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "b9009ded3f80dd6165062654df22855c"
server_key_iv: "45c91225e10a6959bf5287d935bbc11a"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.540162 27615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:37.556318 27615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:37.556619 27615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:37.556828 27615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:37.557049 27615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:37.557386 27615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:37.557442 27615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.557482 27615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:37.557509 27615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.563323 27615 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.2:45869
I20260619 02:44:37.563378 27742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.2:45869 every 8 connection(s)
I20260619 02:44:37.563678 27615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/data/info.pb
I20260619 02:44:37.569034 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.564339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:50879 (local address 127.7.137.62:35749)
0619 02:44:37.564417 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.564419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.564650 (+ 231us) server_negotiation.cc:407] Connection header received
0619 02:44:37.565214 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.565215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.565228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.565282 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:37.565967 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.566483 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.567472 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.567691 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.567768 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.568678 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:37.568679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:37.568679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:37.568684 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:37.568698 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:37.568703 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:37.568785 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.568930 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.568957 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:44:37.569213 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.564334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:35749 (local address 127.7.137.2:50879)
0619 02:44:37.564529 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.564562 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.565101 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.565351 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.565355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.565365 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:37.565866 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.565870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.566580 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.566581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.567372 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.567374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.567775 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.567776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.567910 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.568526 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:37.568526 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:37.568554 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:37.568576 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:37.568788 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:37.568793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:37.568796 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.568886 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.568988 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260619 02:44:37.569654 27743 heartbeater.cc:347] Connected to a master server at 127.7.137.62:35749
I20260619 02:44:37.569746 27743 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:37.569880 27743 heartbeater.cc:511] Master 127.7.137.62:35749 requested a full tablet report, sending...
I20260619 02:44:37.570117 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27615
I20260619 02:44:37.570232 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal/instance
I20260619 02:44:37.570299 27424 ts_manager.cc:194] Registered new tserver with Master: b45722a1d30a4679a447e47480721233 (127.7.137.2:45869)
I20260619 02:44:37.570508 7716 external_mini_cluster.cc:1428] Setting key 932ab7c715aaf74b4f2c0c7ef508af76
I20260619 02:44:37.570763 27424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.2:50879
I20260619 02:44:37.571611 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.137.3:0
--local_ip_for_outbound_sockets=127.7.137.3
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--builtin_ntp_servers=127.7.137.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:44:37.651057 27746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:37.651252 27746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:37.651273 27746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:37.652338 27746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:37.652370 27746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:37.652395 27746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.137.3
I20260619 02:44:37.653712 27746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.137.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.137.3
--webserver_port=0
--tserver_master_addrs=127.7.137.62:35749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.137.3
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:37.653894 27746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:37.654134 27746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:37.659174 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:37.659180 27754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:37.659240 27746 server_base.cc:1034] running on GCE node
W20260619 02:44:37.659255 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:37.659575 27746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:37.659777 27746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:37.660961 27746 hybrid_clock.cc:648] HybridClock initialized: now 1781837077660944 us; error 29 us; skew 500 ppm
I20260619 02:44:37.662134 27746 webserver.cc:460] Webserver started at http://127.7.137.3:42127/ using document root <none> and password file <none>
I20260619 02:44:37.662381 27746 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:37.662448 27746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:37.662599 27746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:37.663642 27746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data/instance:
uuid: "648a80fdcf244f2895501d39a820c967"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "1f7f5a5a6d708e2eb944a45770323722"
server_key_iv: "5030b8fd76ec3aa0f0be5181a35e5798"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.664000 27746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance:
uuid: "648a80fdcf244f2895501d39a820c967"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "1f7f5a5a6d708e2eb944a45770323722"
server_key_iv: "5030b8fd76ec3aa0f0be5181a35e5798"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.665336 27746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:37.666105 27760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.666275 27746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:37.666368 27746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal
uuid: "648a80fdcf244f2895501d39a820c967"
format_stamp: "Formatted at 2026-06-19 02:44:37 on dist-test-slave-pdnb"
server_key: "1f7f5a5a6d708e2eb944a45770323722"
server_key_iv: "5030b8fd76ec3aa0f0be5181a35e5798"
server_key_version: "encryptionkey@0"
I20260619 02:44:37.666461 27746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:37.683979 27746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:37.684283 27746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:37.684428 27746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:37.684641 27746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:44:37.684985 27746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:44:37.685040 27746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.685079 27746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:44:37.685106 27746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:37.690690 27746 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.3:34303
I20260619 02:44:37.690780 27873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.3:34303 every 8 connection(s)
I20260619 02:44:37.691082 27746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/data/info.pb
I20260619 02:44:37.695780 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.691806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.3:47897 (local address 127.7.137.62:35749)
0619 02:44:37.691887 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.691889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.692071 (+ 182us) server_negotiation.cc:407] Connection header received
0619 02:44:37.692522 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.692523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.692534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.692585 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:37.693255 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.693744 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.694666 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.694840 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.694889 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.695578 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:37.695579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:37.695580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:37.695584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:37.695597 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:37.695600 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:37.695647 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.695699 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.695723 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:44:37.696022 27876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.691774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:35749 (local address 127.7.137.3:47897)
0619 02:44:37.691953 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.691985 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.692442 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.692637 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.692640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.692648 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:44:37.693189 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.693194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.693821 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.693822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.694594 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.694596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.694907 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.694908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.695011 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.695459 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:44:37.695459 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:44:37.695485 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:44:37.695505 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:44:37.695652 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:44:37.695655 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:44:37.695656 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.695702 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.695773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260619 02:44:37.696365 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27746
I20260619 02:44:37.696472 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal/instance
I20260619 02:44:37.696571 27874 heartbeater.cc:347] Connected to a master server at 127.7.137.62:35749
I20260619 02:44:37.696664 27874 heartbeater.cc:464] Registering TS with master...
I20260619 02:44:37.696686 7716 external_mini_cluster.cc:1428] Setting key 35557070475aa404936e8e7d5a181d08
I20260619 02:44:37.696866 27874 heartbeater.cc:511] Master 127.7.137.62:35749 requested a full tablet report, sending...
I20260619 02:44:37.697268 27423 ts_manager.cc:194] Registered new tserver with Master: 648a80fdcf244f2895501d39a820c967 (127.7.137.3:34303)
I20260619 02:44:37.697649 27423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.137.3:47897
I20260619 02:44:37.697642 7716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:44:37.701787 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.698946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.7.137.62:35749)
0619 02:44:37.698983 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.698985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.699137 (+ 152us) server_negotiation.cc:407] Connection header received
0619 02:44:37.699215 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.699215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.699228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.699282 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:37.699456 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.699917 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.700649 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.700919 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.700995 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.701548 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:37.701549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:37.701549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:37.701554 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:37.701566 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:37.701570 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:37.701624 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.701683 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.701704 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:37.703150 27423 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:44:37.704555 27423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:44:37.711072 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.707006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.7.137.1:33495)
0619 02:44:37.707423 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.707426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.707589 (+ 163us) server_negotiation.cc:407] Connection header received
0619 02:44:37.707635 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.707637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.707653 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.707728 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.707839 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.708958 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710164 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.710660 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710758 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.710848 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.710939 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.710980 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":111,"threads_started":1}
I20260619 02:44:37.711092 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.706870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33495 (local address 127.0.0.1:47354)
0619 02:44:37.707510 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.707520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.707574 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.707734 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.707735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.707736 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.707775 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.707777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.709050 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.709051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.710085 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.710087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.710761 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.710836 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.710848 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.710922 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.710989 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260619 02:44:37.711251 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.707052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:34303 (local address 127.0.0.1:52618)
0619 02:44:37.707654 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.707661 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.707700 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.707908 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.707909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.707911 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.707947 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.707948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.709169 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.709170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.710109 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.710112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.711051 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.711052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.711105 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.711109 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.711155 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.711190 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260619 02:44:37.711380 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.706792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.7.137.2:45869)
0619 02:44:37.707241 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.707246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.707258 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:44:37.707301 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.707303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.707326 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.707411 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.707808 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.708580 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.709994 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.710729 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.711128 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.711213 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.711237 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.711276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":110,"threads_started":1}
I20260619 02:44:37.711094 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.706667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.2:45869 (local address 127.0.0.1:59392)
0619 02:44:37.706794 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.706832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.706898 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.707437 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.707439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.707442 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.707507 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.707509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.708666 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.708667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.709909 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.709911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.710832 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.710894 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.710899 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.710930 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.710996 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":21,"thread_start_us":65,"threads_started":1}
I20260619 02:44:37.711519 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.707156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.7.137.3:34303)
0619 02:44:37.707327 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.707331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.707704 (+ 373us) server_negotiation.cc:407] Connection header received
0619 02:44:37.707765 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.707767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.707782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.707852 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.708026 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.709085 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710203 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.710881 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.710942 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.711019 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.711344 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.711385 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260619 02:44:37.712105 27677 tablet_service.cc:1467] Processing CreateTablet for tablet a725553088ea427db1e9c38fbbe96e5d (DEFAULT_TABLE table=test-table [id=c84275ea603840f6ad3d2731d5f792e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:37.712136 27808 tablet_service.cc:1467] Processing CreateTablet for tablet a725553088ea427db1e9c38fbbe96e5d (DEFAULT_TABLE table=test-table [id=c84275ea603840f6ad3d2731d5f792e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:37.712440 27677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a725553088ea427db1e9c38fbbe96e5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:37.712451 27808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a725553088ea427db1e9c38fbbe96e5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:37.713321 27546 tablet_service.cc:1467] Processing CreateTablet for tablet a725553088ea427db1e9c38fbbe96e5d (DEFAULT_TABLE table=test-table [id=c84275ea603840f6ad3d2731d5f792e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:44:37.713639 27546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a725553088ea427db1e9c38fbbe96e5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:37.715304 27888 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Bootstrap starting.
I20260619 02:44:37.715878 27888 tablet_bootstrap.cc:654] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:37.716207 27889 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Bootstrap starting.
I20260619 02:44:37.716249 27888 log.cc:824] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:37.716878 27891 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: Bootstrap starting.
I20260619 02:44:37.716917 27889 tablet_bootstrap.cc:654] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:37.717191 27888 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: No bootstrap required, opened a new log
I20260619 02:44:37.717224 27889 log.cc:824] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:37.717258 27888 ts_tablet_manager.cc:1397] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:44:37.717562 27891 tablet_bootstrap.cc:654] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:37.717851 27891 log.cc:824] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:37.717999 27889 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: No bootstrap required, opened a new log
I20260619 02:44:37.718068 27889 ts_tablet_manager.cc:1397] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:44:37.719930 27888 raft_consensus.cc:348] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.720118 27888 raft_consensus.cc:374] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:37.720155 27888 raft_consensus.cc:729] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648a80fdcf244f2895501d39a820c967, State: Initialized, Role: FOLLOWER
I20260619 02:44:37.720317 27888 consensus_queue.cc:260] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.720593 27888 ts_tablet_manager.cc:1428] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:44:37.720580 27889 raft_consensus.cc:348] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.720757 27874 heartbeater.cc:503] Master 127.7.137.62:35749 was elected leader, sending a full tablet report...
I20260619 02:44:37.720767 27889 raft_consensus.cc:374] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:37.720806 27889 raft_consensus.cc:729] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45722a1d30a4679a447e47480721233, State: Initialized, Role: FOLLOWER
I20260619 02:44:37.720956 27889 consensus_queue.cc:260] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.721235 27889 ts_tablet_manager.cc:1428] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 02:44:37.721264 27743 heartbeater.cc:503] Master 127.7.137.62:35749 was elected leader, sending a full tablet report...
I20260619 02:44:37.722010 27891 tablet_bootstrap.cc:492] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: No bootstrap required, opened a new log
I20260619 02:44:37.722083 27891 ts_tablet_manager.cc:1397] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 02:44:37.723896 27891 raft_consensus.cc:348] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.724020 27891 raft_consensus.cc:374] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:37.724041 27891 raft_consensus.cc:729] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3a602937334054a4892d09408b73bf, State: Initialized, Role: FOLLOWER
I20260619 02:44:37.724171 27891 consensus_queue.cc:260] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.724412 27891 ts_tablet_manager.cc:1428] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:44:37.724468 27612 heartbeater.cc:503] Master 127.7.137.62:35749 was elected leader, sending a full tablet report...
W20260619 02:44:37.814189 27744 tablet.cc:2369] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:37.884704 27895 raft_consensus.cc:482] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:44:37.884857 27895 raft_consensus.cc:504] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.885201 27895 leader_election.cc:288] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3a602937334054a4892d09408b73bf (127.7.137.1:33495), 648a80fdcf244f2895501d39a820c967 (127.7.137.3:34303)
I20260619 02:44:37.889222 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.885629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:57769 (local address 127.7.137.3:34303)
0619 02:44:37.885670 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.885671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.885780 (+ 109us) server_negotiation.cc:407] Connection header received
0619 02:44:37.885820 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.885820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.885828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.885897 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.886121 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.886805 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888243 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.888818 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888888 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.888898 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.889109 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.889149 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:37.889222 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.885484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.137.2:34863 (local address 127.7.137.1:33495)
0619 02:44:37.885560 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.885562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.885575 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:44:37.885607 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.885608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.885620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.885695 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:44:37.885895 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.886714 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888243 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.888815 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.888898 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.889109 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.889150 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:44:37.889235 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.885391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33495 (local address 127.7.137.2:34863)
0619 02:44:37.885458 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.885471 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.885520 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.885716 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.885717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.885719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.885795 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.885797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.886830 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.886831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.888146 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.888149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.888927 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.889014 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.889019 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.889087 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.889135 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":43}
I20260619 02:44:37.889241 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.885522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.3:34303 (local address 127.7.137.2:57769)
0619 02:44:37.885702 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.885712 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:44:37.885767 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:44:37.885918 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:44:37.885919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:44:37.885922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:44:37.885981 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.885982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.886908 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.886908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.888147 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:44:37.888149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.888939 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.888940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:44:37.889014 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.889020 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:44:37.889087 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:44:37.889134 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":29,"thread_start_us":129,"threads_started":1}
I20260619 02:44:37.889714 27828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a725553088ea427db1e9c38fbbe96e5d" candidate_uuid: "b45722a1d30a4679a447e47480721233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648a80fdcf244f2895501d39a820c967" is_pre_election: true
I20260619 02:44:37.889716 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a725553088ea427db1e9c38fbbe96e5d" candidate_uuid: "b45722a1d30a4679a447e47480721233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3a602937334054a4892d09408b73bf" is_pre_election: true
I20260619 02:44:37.889876 27566 raft_consensus.cc:2454] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45722a1d30a4679a447e47480721233 in term 0.
I20260619 02:44:37.889875 27828 raft_consensus.cc:2454] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45722a1d30a4679a447e47480721233 in term 0.
I20260619 02:44:37.890110 27630 leader_election.cc:302] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648a80fdcf244f2895501d39a820c967, b45722a1d30a4679a447e47480721233; no voters:
I20260619 02:44:37.890236 27895 raft_consensus.cc:2781] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:44:37.890290 27895 raft_consensus.cc:482] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:44:37.890322 27895 raft_consensus.cc:3037] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.891026 27895 raft_consensus.cc:504] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.891167 27895 leader_election.cc:288] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [CANDIDATE]: Term 1 election: Requested vote from peers fe3a602937334054a4892d09408b73bf (127.7.137.1:33495), 648a80fdcf244f2895501d39a820c967 (127.7.137.3:34303)
I20260619 02:44:37.891376 27828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a725553088ea427db1e9c38fbbe96e5d" candidate_uuid: "b45722a1d30a4679a447e47480721233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648a80fdcf244f2895501d39a820c967"
I20260619 02:44:37.891403 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a725553088ea427db1e9c38fbbe96e5d" candidate_uuid: "b45722a1d30a4679a447e47480721233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3a602937334054a4892d09408b73bf"
I20260619 02:44:37.891462 27828 raft_consensus.cc:3037] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.891494 27566 raft_consensus.cc:3037] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:37.892190 27828 raft_consensus.cc:2454] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45722a1d30a4679a447e47480721233 in term 1.
I20260619 02:44:37.892287 27566 raft_consensus.cc:2454] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45722a1d30a4679a447e47480721233 in term 1.
I20260619 02:44:37.892413 27630 leader_election.cc:302] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648a80fdcf244f2895501d39a820c967, b45722a1d30a4679a447e47480721233; no voters:
I20260619 02:44:37.892565 27895 raft_consensus.cc:2781] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:37.892766 27895 raft_consensus.cc:686] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 1 LEADER]: Becoming Leader. State: Replica: b45722a1d30a4679a447e47480721233, State: Running, Role: LEADER
I20260619 02:44:37.892910 27895 consensus_queue.cc:237] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } }
I20260619 02:44:37.893771 27423 catalog_manager.cc:5495] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 reported cstate change: term changed from 0 to 1, leader changed from <none> to b45722a1d30a4679a447e47480721233 (127.7.137.2). New cstate: current_term: 1 leader_uuid: "b45722a1d30a4679a447e47480721233" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45722a1d30a4679a447e47480721233" member_type: VOTER last_known_addr { host: "127.7.137.2" port: 45869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 } health_report { overall_health: UNKNOWN } } }
I20260619 02:44:37.935453 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.933242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.7.137.2:45869)
0619 02:44:37.933301 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:37.933302 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:37.933313 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:44:37.933357 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:37.933358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:37.933374 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:37.933434 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:44:37.933563 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.934010 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.934760 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:37.934933 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:37.934980 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:37.935160 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:44:37.935254 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:44:37.935313 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:37.935364 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:44:37.935378 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:44:37.937400 27566 raft_consensus.cc:1261] T a725553088ea427db1e9c38fbbe96e5d P fe3a602937334054a4892d09408b73bf [term 1 FOLLOWER]: Refusing update from remote peer b45722a1d30a4679a447e47480721233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:37.937400 27828 raft_consensus.cc:1261] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 1 FOLLOWER]: Refusing update from remote peer b45722a1d30a4679a447e47480721233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:44:37.937786 27898 consensus_queue.cc:1035] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648a80fdcf244f2895501d39a820c967" member_type: VOTER last_known_addr { host: "127.7.137.3" port: 34303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:37.937948 27895 consensus_queue.cc:1035] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3a602937334054a4892d09408b73bf" member_type: VOTER last_known_addr { host: "127.7.137.1" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:44:37.941234 27902 mvcc.cc:205] Tried to move back new op lower bound from 7298404671228456960 to 7298404671050039296. Current Snapshot: MvccSnapshot[applied={T|T < 7298404671228456960 or (T in {7298404671228456960})}]
W20260619 02:44:37.941552 27875 tablet.cc:2369] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:44:37.942646 27423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:37.942739 27423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:44:37.943467 27423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc50b781812c45cc92f4682201d937c6: Sending DeleteTablet for 3 replicas of tablet a725553088ea427db1e9c38fbbe96e5d
I20260619 02:44:37.943843 27546 tablet_service.cc:1514] Processing DeleteTablet for tablet a725553088ea427db1e9c38fbbe96e5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='slave'} at 127.0.0.1:47354
I20260619 02:44:37.943866 27808 tablet_service.cc:1514] Processing DeleteTablet for tablet a725553088ea427db1e9c38fbbe96e5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='slave'} at 127.0.0.1:52618
I20260619 02:44:37.944056 27807 tablet_service.cc:1514] Processing DeleteTablet for tablet a725553088ea427db1e9c38fbbe96e5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='slave'} at 127.0.0.1:52618
I20260619 02:44:37.944121 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27484
I20260619 02:44:37.944240 27677 tablet_service.cc:1514] Processing DeleteTablet for tablet a725553088ea427db1e9c38fbbe96e5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:44:37 UTC) from {username='slave'} at 127.0.0.1:59392
I20260619 02:44:37.945016 27914 tablet_replica.cc:330] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: stopping tablet replica
I20260619 02:44:37.945137 27914 raft_consensus.cc:2229] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:44:37.945253 27914 raft_consensus.cc:2258] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:37.945622 27914 ts_tablet_manager.cc:1905] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:37.947242 27914 ts_tablet_manager.cc:1918] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:37.947393 27914 log.cc:1196] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-1/wal/wals/a725553088ea427db1e9c38fbbe96e5d
I20260619 02:44:37.947738 27914 ts_tablet_manager.cc:1939] T a725553088ea427db1e9c38fbbe96e5d P b45722a1d30a4679a447e47480721233: Deleting consensus metadata
I20260619 02:44:37.948127 27408 catalog_manager.cc:4841] TS b45722a1d30a4679a447e47480721233 (127.7.137.2:45869): tablet a725553088ea427db1e9c38fbbe96e5d (table test-table [id=c84275ea603840f6ad3d2731d5f792e9]) successfully deleted
I20260619 02:44:37.948166 27913 tablet_replica.cc:330] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: stopping tablet replica
I20260619 02:44:37.948261 27913 raft_consensus.cc:2229] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:44:37.948362 27913 raft_consensus.cc:2258] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:44:37.948829 27913 ts_tablet_manager.cc:1905] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:44:37.950026 27913 ts_tablet_manager.cc:1918] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:44:37.950111 27913 log.cc:1196] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Deleting WAL directory at /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836839601721-7716-0/minicluster-data/ts-2/wal/wals/a725553088ea427db1e9c38fbbe96e5d
I20260619 02:44:37.950344 27913 ts_tablet_manager.cc:1939] T a725553088ea427db1e9c38fbbe96e5d P 648a80fdcf244f2895501d39a820c967: Deleting consensus metadata
I20260619 02:44:37.950753 27408 catalog_manager.cc:4841] TS 648a80fdcf244f2895501d39a820c967 (127.7.137.3:34303): tablet a725553088ea427db1e9c38fbbe96e5d (table test-table [id=c84275ea603840f6ad3d2731d5f792e9]) successfully deleted
W20260619 02:44:37.950894 27408 catalog_manager.cc:4808] TS 648a80fdcf244f2895501d39a820c967 (127.7.137.3:34303): delete failed for tablet a725553088ea427db1e9c38fbbe96e5d because the tablet was not found. No further retry: Not found: Tablet not found: a725553088ea427db1e9c38fbbe96e5d
W20260619 02:44:37.952525 27408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.137.1:33495 (error 108)
I20260619 02:44:37.952665 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27615
I20260619 02:44:37.954067 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:37.953913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.1:33495 (local address 127.0.0.1:47360)
0619 02:44:37.953981 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:44:37.954040 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:44:37.954195 27408 catalog_manager.cc:4584] TS fe3a602937334054a4892d09408b73bf (127.7.137.1:33495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a725553088ea427db1e9c38fbbe96e5d: Network error: Client connection negotiation failed: client connection to 127.7.137.1:33495: connect: Connection refused (error 111)
I20260619 02:44:37.958595 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27746
I20260619 02:44:37.962994 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27393
2026-06-19T02:44:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (838 ms)
[----------] 6 tests from EncryptionPolicyTest (8593 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:38 dist-test-slave-pdnb krb5kdc[27918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:38 dist-test-slave-pdnb krb5kdc[27918](info): set up 2 sockets
Jun 19 02:44:38 dist-test-slave-pdnb krb5kdc[27918](info): commencing operation
krb5kdc: starting...
W20260619 02:44:39.024680 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[27918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:39.139415 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:39.225937 27931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:39.226198 27931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:39.226239 27931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:39.227545 27931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:39.227624 27931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:39.227653 27931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:39.227685 27931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:39.227717 27931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:39.229350 27931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:39.229664 27931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:39.229915 27931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:44:39.234777 27931 server_base.cc:1034] running on GCE node
W20260619 02:44:39.234752 27939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:39.234865 27936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:39.234791 27937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:39.235373 27931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:39.235651 27931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:39.236799 27931 hybrid_clock.cc:648] HybridClock initialized: now 1781837079236775 us; error 47 us; skew 500 ppm
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[27918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837079, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:39.239174 27931 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:39.239601 27931 webserver.cc:460] Webserver started at http://127.0.0.1:46309/ using document root <none> and password file <none>
I20260619 02:44:39.239792 27931 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:39.239840 27931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:39.239993 27931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:39.241037 27931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "fa36c218b0f74832b369f6242aad7568"
format_stamp: "Formatted at 2026-06-19 02:44:39 on dist-test-slave-pdnb"
server_key: "10b21fdb39acd035516cfd93e898de81"
server_key_iv: "53ce7d185b8a9c29f9b65f1b5eb4a35a"
server_key_version: "encryptionkey@0"
I20260619 02:44:39.241362 27931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "fa36c218b0f74832b369f6242aad7568"
format_stamp: "Formatted at 2026-06-19 02:44:39 on dist-test-slave-pdnb"
server_key: "10b21fdb39acd035516cfd93e898de81"
server_key_iv: "53ce7d185b8a9c29f9b65f1b5eb4a35a"
server_key_version: "encryptionkey@0"
I20260619 02:44:39.242554 27931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:39.243256 27946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:39.243423 27931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:39.243530 27931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "fa36c218b0f74832b369f6242aad7568"
format_stamp: "Formatted at 2026-06-19 02:44:39 on dist-test-slave-pdnb"
server_key: "10b21fdb39acd035516cfd93e898de81"
server_key_iv: "53ce7d185b8a9c29f9b65f1b5eb4a35a"
server_key_version: "encryptionkey@0"
I20260619 02:44:39.243603 27931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:39.258977 27931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:39.260998 27931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:39.261153 27931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:39.265221 27931 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34001
I20260619 02:44:39.265264 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34001 every 8 connection(s)
I20260619 02:44:39.265617 27931 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:39.266559 27999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:39.268795 27999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568: Bootstrap starting.
I20260619 02:44:39.269362 27999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:39.269649 27999 log.cc:824] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:39.270303 27999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568: No bootstrap required, opened a new log
I20260619 02:44:39.271847 27999 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260619 02:44:39.271972 27999 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:39.271994 27999 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa36c218b0f74832b369f6242aad7568, State: Initialized, Role: FOLLOWER
I20260619 02:44:39.272136 27999 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260619 02:44:39.272212 27999 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:39.272234 27999 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:39.272302 27999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:39.272859 27999 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260619 02:44:39.272940 27999 leader_election.cc:302] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa36c218b0f74832b369f6242aad7568; no voters:
I20260619 02:44:39.273136 27999 leader_election.cc:288] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:39.273167 28002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:39.273422 27999 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:39.273442 28002 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [term 1 LEADER]: Becoming Leader. State: Replica: fa36c218b0f74832b369f6242aad7568, State: Running, Role: LEADER
I20260619 02:44:39.273557 28002 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260619 02:44:39.274107 28003 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa36c218b0f74832b369f6242aad7568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260619 02:44:39.274223 28003 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:39.274425 28004 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa36c218b0f74832b369f6242aad7568. Latest consensus state: current_term: 1 leader_uuid: "fa36c218b0f74832b369f6242aad7568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa36c218b0f74832b369f6242aad7568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260619 02:44:39.274513 28004 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:39.274705 28011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:39.274804 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 27931
I20260619 02:44:39.274926 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:39.275148 7716 external_mini_cluster.cc:1428] Setting key 3a9835f11386fa1f7b46d7b9c2b2f4ab
I20260619 02:44:39.275208 28011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:39.276820 28011 catalog_manager.cc:1347] Generated new cluster ID: b330ef354fbf4b6898f8caee912a7e17
I20260619 02:44:39.276885 28011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:39.282593 28011 catalog_manager.cc:1370] Generated new certificate authority record
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[27918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:44:39.283272 28011 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:39.287012 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:39.275671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40175 (local address 127.0.0.1:34001)
0619 02:44:39.276164 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:39.276172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:39.276197 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:44:39.276370 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:39.276378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:39.276449 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:39.276611 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:39.277488 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:39.278904 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:39.280044 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:39.280669 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:39.280830 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:39.283663 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:39.283667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:39.283667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:39.283679 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:39.285420 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:39.285813 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:39.285817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:39.285817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:39.285910 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:39.286140 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:39.286141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:39.286141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:39.286368 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:39.286547 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:39.286632 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:44:39.286834 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":81,"threads_started":1}
I20260619 02:44:39.287933 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:39.292943 28011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa36c218b0f74832b369f6242aad7568: Generated new TSK 0
I20260619 02:44:39.293112 28011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:39.299695 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:39.289346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35597 (local address 127.0.0.1:34001)
0619 02:44:39.289383 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:39.289385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:39.292773 (+ 3388us) server_negotiation.cc:407] Connection header received
0619 02:44:39.292800 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:39.292801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:39.292813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:39.292950 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:39.293592 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:39.294157 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:39.294995 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:39.295238 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:39.295339 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:39.296644 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:39.296645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:39.296646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:39.296654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:39.298292 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:39.298688 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:39.298690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:39.298690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:39.298748 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:39.299002 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:39.299003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:39.299004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:39.299121 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:39.299243 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:39.299533 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 02:44:39.299597 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:39.301718 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 27931
W20260619 02:44:39.305543 27916 connection.cc:535] client connection to 127.0.0.1:34001 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34001): Connection reset by peer (error 104)
2026-06-19T02:44:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[28030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[28030](info): set up 2 sockets
Jun 19 02:44:39 dist-test-slave-pdnb krb5kdc[28030](info): commencing operation
krb5kdc: starting...
W20260619 02:44:40.336921 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:40 dist-test-slave-pdnb krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:40.455514 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:40.536239 28043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:40.536446 28043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:40.536468 28043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:40.537680 28043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:40.537724 28043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:40.537739 28043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:40.537750 28043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:40.537786 28043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:40.539243 28043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:40.539492 28043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:40.539752 28043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:40.544795 28048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:40.544799 28051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:40.544817 28043 server_base.cc:1034] running on GCE node
W20260619 02:44:40.544809 28049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:40.545244 28043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:40.545488 28043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:40.546630 28043 hybrid_clock.cc:648] HybridClock initialized: now 1781837080546617 us; error 27 us; skew 500 ppm
Jun 19 02:44:40 dist-test-slave-pdnb krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837080, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:40.553679 28043 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:40.554157 28043 webserver.cc:460] Webserver started at http://127.0.0.1:34467/ using document root <none> and password file <none>
I20260619 02:44:40.554389 28043 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:40.554457 28043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:40.554615 28043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:40.555675 28043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "8f447767247946c083a269aff204931a"
format_stamp: "Formatted at 2026-06-19 02:44:40 on dist-test-slave-pdnb"
server_key: "005deaad29d3af36c8a79dd6430f37e0"
server_key_iv: "7f24360e7972688ecac52688ae9b1ab8"
server_key_version: "encryptionkey@0"
I20260619 02:44:40.556038 28043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "8f447767247946c083a269aff204931a"
format_stamp: "Formatted at 2026-06-19 02:44:40 on dist-test-slave-pdnb"
server_key: "005deaad29d3af36c8a79dd6430f37e0"
server_key_iv: "7f24360e7972688ecac52688ae9b1ab8"
server_key_version: "encryptionkey@0"
I20260619 02:44:40.557512 28043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:40.558384 28058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:40.558614 28043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:40.558729 28043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "8f447767247946c083a269aff204931a"
format_stamp: "Formatted at 2026-06-19 02:44:40 on dist-test-slave-pdnb"
server_key: "005deaad29d3af36c8a79dd6430f37e0"
server_key_iv: "7f24360e7972688ecac52688ae9b1ab8"
server_key_version: "encryptionkey@0"
I20260619 02:44:40.558804 28043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:40.584647 28043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:40.586586 28043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:40.586750 28043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:40.590960 28043 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39149
I20260619 02:44:40.591013 28110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39149 every 8 connection(s)
I20260619 02:44:40.591392 28043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:40.592360 28111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:40.594627 28111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a: Bootstrap starting.
I20260619 02:44:40.595139 28111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:40.595364 28111 log.cc:824] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:40.595988 28111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a: No bootstrap required, opened a new log
I20260619 02:44:40.597540 28111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260619 02:44:40.597673 28111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:40.597695 28111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f447767247946c083a269aff204931a, State: Initialized, Role: FOLLOWER
I20260619 02:44:40.597832 28111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260619 02:44:40.597918 28111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:40.597942 28111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:40.597977 28111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:40.598444 28111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260619 02:44:40.598526 28111 leader_election.cc:302] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f447767247946c083a269aff204931a; no voters:
I20260619 02:44:40.598642 28111 leader_election.cc:288] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:40.598691 28114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:40.598860 28114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [term 1 LEADER]: Becoming Leader. State: Replica: 8f447767247946c083a269aff204931a, State: Running, Role: LEADER
I20260619 02:44:40.598924 28111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:40.598994 28114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260619 02:44:40.599409 28116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f447767247946c083a269aff204931a. Latest consensus state: current_term: 1 leader_uuid: "8f447767247946c083a269aff204931a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } } }
I20260619 02:44:40.599541 28116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:40.599543 28115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f447767247946c083a269aff204931a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f447767247946c083a269aff204931a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } } }
I20260619 02:44:40.599871 28115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:40.599941 28123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:40.600451 28123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:40.600605 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28043
I20260619 02:44:40.600737 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:40.600926 7716 external_mini_cluster.cc:1428] Setting key 2a77c08703f9851ce28db7fc69251dca
I20260619 02:44:40.602154 28123 catalog_manager.cc:1347] Generated new cluster ID: e5e0574c261f457d8d7e79b23683cbec
I20260619 02:44:40.602206 28123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:40.609573 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:40.601396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37955 (local address 127.0.0.1:39149)
0619 02:44:40.601544 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:40.601553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:40.601590 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:44:40.601707 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:40.601714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:40.601784 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:40.601928 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:40.604895 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:40.606346 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:40.607396 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:40.608017 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:40.608130 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:40.608877 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:40.608882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:40.608882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:40.609096 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:40.609140 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:40.609156 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:40.609246 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:40.609327 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:44:40.609355 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260619 02:44:40.610522 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:40.616037 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:40.611995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46529 (local address 127.0.0.1:39149)
0619 02:44:40.612788 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:40.612790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:40.612798 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:40.612817 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:40.612818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:40.612827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:40.612884 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:40.613037 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:40.613789 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:40.614730 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:40.615069 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:40.615140 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:40.615798 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:40.615799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:40.615800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:40.615804 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:40.615818 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:40.615822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:40.615884 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:40.615952 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 02:44:40.615963 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":773}
I20260619 02:44:40.624883 28123 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:40.625370 28123 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:40.628046 28123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f447767247946c083a269aff204931a: Generated new TSK 0
I20260619 02:44:40.628207 28123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:40.632386 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28043
W20260619 02:44:40.636380 28028 connection.cc:535] client connection to 127.0.0.1:39149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39149): Connection reset by peer (error 104)
2026-06-19T02:44:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:40 dist-test-slave-pdnb krb5kdc[28142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:40 dist-test-slave-pdnb krb5kdc[28142](info): set up 2 sockets
Jun 19 02:44:40 dist-test-slave-pdnb krb5kdc[28142](info): commencing operation
krb5kdc: starting...
W20260619 02:44:41.701133 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:41 dist-test-slave-pdnb krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:41.821630 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:41.906677 28155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:41.906890 28155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:41.906921 28155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:41.908116 28155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:41.908151 28155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:41.908166 28155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:41.908179 28155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:41.908190 28155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:41.909653 28155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:41.909833 28155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:41.909994 28155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:41.915853 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:41.915867 28155 server_base.cc:1034] running on GCE node
W20260619 02:44:41.915843 28161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:41.915869 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:41.916312 28155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:41.916605 28155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:41.917819 28155 hybrid_clock.cc:648] HybridClock initialized: now 1781837081917797 us; error 49 us; skew 500 ppm
Jun 19 02:44:41 dist-test-slave-pdnb krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837081, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:41.920389 28155 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:41.920995 28155 webserver.cc:460] Webserver started at http://127.0.0.1:38123/ using document root <none> and password file <none>
I20260619 02:44:41.921273 28155 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:41.921348 28155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:41.921511 28155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:41.922626 28155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "42457b8a95904ea0b23881d069357975"
format_stamp: "Formatted at 2026-06-19 02:44:41 on dist-test-slave-pdnb"
server_key: "aa0b9caf3e132ef29faf8d3f8d5f13bb"
server_key_iv: "b866f40f5ff807e5791e606b55b98cbe"
server_key_version: "encryptionkey@0"
I20260619 02:44:41.923018 28155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "42457b8a95904ea0b23881d069357975"
format_stamp: "Formatted at 2026-06-19 02:44:41 on dist-test-slave-pdnb"
server_key: "aa0b9caf3e132ef29faf8d3f8d5f13bb"
server_key_iv: "b866f40f5ff807e5791e606b55b98cbe"
server_key_version: "encryptionkey@0"
I20260619 02:44:41.924386 28155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:41.925238 28170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:41.925450 28155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:44:41.925565 28155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "42457b8a95904ea0b23881d069357975"
format_stamp: "Formatted at 2026-06-19 02:44:41 on dist-test-slave-pdnb"
server_key: "aa0b9caf3e132ef29faf8d3f8d5f13bb"
server_key_iv: "b866f40f5ff807e5791e606b55b98cbe"
server_key_version: "encryptionkey@0"
I20260619 02:44:41.925644 28155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:41.929302 28155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:41.931389 28155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:41.931556 28155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:41.935588 28155 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45709
I20260619 02:44:41.935616 28222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45709 every 8 connection(s)
I20260619 02:44:41.935957 28155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:41.936921 28223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:41.939023 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975: Bootstrap starting.
I20260619 02:44:41.939594 28223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:41.939867 28223 log.cc:824] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:41.940544 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975: No bootstrap required, opened a new log
I20260619 02:44:41.942124 28223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } }
I20260619 02:44:41.942251 28223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:41.942273 28223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42457b8a95904ea0b23881d069357975, State: Initialized, Role: FOLLOWER
I20260619 02:44:41.942420 28223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } }
I20260619 02:44:41.942481 28223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:41.942502 28223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:41.942570 28223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:41.943075 28223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } }
I20260619 02:44:41.943152 28223 leader_election.cc:302] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42457b8a95904ea0b23881d069357975; no voters:
I20260619 02:44:41.943341 28223 leader_election.cc:288] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:41.943375 28226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:41.943584 28226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [term 1 LEADER]: Becoming Leader. State: Replica: 42457b8a95904ea0b23881d069357975, State: Running, Role: LEADER
I20260619 02:44:41.943681 28226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } }
I20260619 02:44:41.943825 28223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:41.944015 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42457b8a95904ea0b23881d069357975. Latest consensus state: current_term: 1 leader_uuid: "42457b8a95904ea0b23881d069357975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } } }
I20260619 02:44:41.944150 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:41.944167 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42457b8a95904ea0b23881d069357975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42457b8a95904ea0b23881d069357975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45709 } } }
I20260619 02:44:41.944311 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:41.944691 28232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:41.945303 28232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:41.945763 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28155
I20260619 02:44:41.945883 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:41.946063 7716 external_mini_cluster.cc:1428] Setting key 8021b685143904d8b585a715a7753991
I20260619 02:44:41.947224 28232 catalog_manager.cc:1347] Generated new cluster ID: c3a919058b1546abb92ea2d370c8958a
I20260619 02:44:41.947273 28232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:41.948205 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:41.946700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50735 (local address 127.0.0.1:45709)
0619 02:44:41.946834 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:41.946842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:41.946878 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:44:41.946964 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:41.946969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:41.946991 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:41.947137 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:41.947795 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:41.947798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:41.947798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:41.947803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:41.947847 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:41.947865 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:41.947937 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:41.948008 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:44:41.948050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260619 02:44:41.949642 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:41.953884 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:41.950916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43835 (local address 127.0.0.1:45709)
0619 02:44:41.950953 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:41.950955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:41.952901 (+ 1946us) server_negotiation.cc:407] Connection header received
0619 02:44:41.952976 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:41.952977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:41.952982 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:41.953031 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:41.953633 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:41.953634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:41.953634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:41.953639 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:41.953652 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:41.953656 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:41.953716 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:41.953772 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 02:44:41.953788 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:44:41.974151 28232 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:41.974339 28232 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:41.978937 28232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42457b8a95904ea0b23881d069357975: Generated new TSK 0
I20260619 02:44:41.979117 28232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:41.987121 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28155
2026-06-19T02:44:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:42 dist-test-slave-pdnb krb5kdc[28254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:42 dist-test-slave-pdnb krb5kdc[28254](info): set up 2 sockets
Jun 19 02:44:42 dist-test-slave-pdnb krb5kdc[28254](info): commencing operation
krb5kdc: starting...
W20260619 02:44:43.032817 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:43.150938 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:43.231068 28267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:43.231292 28267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:43.231315 28267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:43.232491 28267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:43.232532 28267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:43.232546 28267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:43.232559 28267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:43.232617 28267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:43.234270 28267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:43.234529 28267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:43.234758 28267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:43.239619 28272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:43.239610 28275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:43.239751 28267 server_base.cc:1034] running on GCE node
W20260619 02:44:43.239682 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:43.240095 28267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:43.240388 28267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:43.241554 28267 hybrid_clock.cc:648] HybridClock initialized: now 1781837083241558 us; error 48 us; skew 500 ppm
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837083, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:43.244028 28267 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:43.244490 28267 webserver.cc:460] Webserver started at http://127.0.0.1:35113/ using document root <none> and password file <none>
I20260619 02:44:43.244784 28267 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:43.244863 28267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:43.245025 28267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:43.246050 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "1857d156166e4ca19dbea182cf24cf52"
format_stamp: "Formatted at 2026-06-19 02:44:43 on dist-test-slave-pdnb"
server_key: "0b3d6dcaac863d97e1df06654ec22472"
server_key_iv: "b2a1c179d1efd86bbfc1afeb2f7a656b"
server_key_version: "encryptionkey@0"
I20260619 02:44:43.246409 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "1857d156166e4ca19dbea182cf24cf52"
format_stamp: "Formatted at 2026-06-19 02:44:43 on dist-test-slave-pdnb"
server_key: "0b3d6dcaac863d97e1df06654ec22472"
server_key_iv: "b2a1c179d1efd86bbfc1afeb2f7a656b"
server_key_version: "encryptionkey@0"
I20260619 02:44:43.247732 28267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:44:43.248417 28282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:43.248584 28267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:43.248673 28267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "1857d156166e4ca19dbea182cf24cf52"
format_stamp: "Formatted at 2026-06-19 02:44:43 on dist-test-slave-pdnb"
server_key: "0b3d6dcaac863d97e1df06654ec22472"
server_key_iv: "b2a1c179d1efd86bbfc1afeb2f7a656b"
server_key_version: "encryptionkey@0"
I20260619 02:44:43.248809 28267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:43.268054 28267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:43.270219 28267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:43.270385 28267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:43.274340 28267 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33569
I20260619 02:44:43.274441 28334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33569 every 8 connection(s)
I20260619 02:44:43.274778 28267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:43.275496 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28267
I20260619 02:44:43.275636 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:43.275810 7716 external_mini_cluster.cc:1428] Setting key 211747e086ac17bdcbf52c4f64e80e58
I20260619 02:44:43.276002 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:43.279469 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52: Bootstrap starting.
I20260619 02:44:43.280085 28335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:43.280413 28335 log.cc:824] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:43.281219 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52: No bootstrap required, opened a new log
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:44:43.283008 28335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260619 02:44:43.283201 28335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:43.283268 28335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1857d156166e4ca19dbea182cf24cf52, State: Initialized, Role: FOLLOWER
I20260619 02:44:43.283411 28335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260619 02:44:43.283496 28335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:43.283540 28335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:43.283596 28335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:43.284363 28335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260619 02:44:43.284479 28335 leader_election.cc:302] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1857d156166e4ca19dbea182cf24cf52; no voters:
I20260619 02:44:43.284648 28335 leader_election.cc:288] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:43.284744 28340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:43.284960 28340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [term 1 LEADER]: Becoming Leader. State: Replica: 1857d156166e4ca19dbea182cf24cf52, State: Running, Role: LEADER
I20260619 02:44:43.285002 28335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:43.285112 28340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260619 02:44:43.285656 28342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1857d156166e4ca19dbea182cf24cf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260619 02:44:43.285667 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1857d156166e4ca19dbea182cf24cf52. Latest consensus state: current_term: 1 leader_uuid: "1857d156166e4ca19dbea182cf24cf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1857d156166e4ca19dbea182cf24cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260619 02:44:43.285781 28342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:43.285791 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:43.286563 28349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:43.287217 28349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:43.287602 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:43.276350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33413 (local address 127.0.0.1:33569)
0619 02:44:43.276579 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:43.276589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:43.276622 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:44:43.277296 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:43.277305 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:43.277368 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:43.277521 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:43.278146 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:43.279512 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:43.280353 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:43.281007 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:43.281119 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:43.283648 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:43.283652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:43.283653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:43.283664 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:43.285837 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:43.286282 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:43.286284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:43.286285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:43.286405 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:43.286640 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:43.286641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:43.286641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:43.286946 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:43.287119 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:43.287215 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:44:43.287446 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":101,"threads_started":1}
I20260619 02:44:43.288509 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:43.288666 28349 catalog_manager.cc:1347] Generated new cluster ID: b342c04ba8f04ee19576f20ffd5504f3
I20260619 02:44:43.288707 28349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:43.297062 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:43.290069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60445 (local address 127.0.0.1:33569)
0619 02:44:43.290105 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:43.290107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:43.290115 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:43.290203 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:43.290204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:43.290214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:43.290279 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:43.290857 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:43.291608 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:43.292575 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:43.292934 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:43.293011 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:43.294304 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:43.294305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:43.294306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:43.294313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:43.295912 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:43.296274 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:43.296275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:43.296276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:43.296337 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:43.296536 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:43.296537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:43.296537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:43.296668 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:43.296809 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:43.296890 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:44:43.296963 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260619 02:44:43.303138 28349 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:43.303864 28349 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:43.309324 28349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1857d156166e4ca19dbea182cf24cf52: Generated new TSK 0
I20260619 02:44:43.309516 28349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:43.316200 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28267
2026-06-19T02:44:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28366](info): set up 2 sockets
Jun 19 02:44:43 dist-test-slave-pdnb krb5kdc[28366](info): commencing operation
krb5kdc: starting...
W20260619 02:44:44.352676 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:44 dist-test-slave-pdnb krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:44.469781 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:44.548352 28379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:44.548578 28379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:44.548614 28379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:44.549774 28379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:44.549840 28379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:44.549865 28379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:44.549896 28379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:44.549924 28379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:44.551343 28379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:44.551605 28379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:44.551822 28379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:44.556516 28387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:44.556489 28385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:44.556589 28379 server_base.cc:1034] running on GCE node
W20260619 02:44:44.556499 28384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:44.556936 28379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:44.557161 28379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:44.558339 28379 hybrid_clock.cc:648] HybridClock initialized: now 1781837084558319 us; error 40 us; skew 500 ppm
Jun 19 02:44:44 dist-test-slave-pdnb krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837084, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:44.560837 28379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:44.561285 28379 webserver.cc:460] Webserver started at http://127.0.0.1:40043/ using document root <none> and password file <none>
I20260619 02:44:44.561523 28379 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:44.561590 28379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:44.561719 28379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:44.562714 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "493967ece2f44162ab9951bc96b514f6"
format_stamp: "Formatted at 2026-06-19 02:44:44 on dist-test-slave-pdnb"
server_key: "73e5c8be8e0e64b7ae9de039b5f9c6ff"
server_key_iv: "fc545718d077834a1343d5323b23a69e"
server_key_version: "encryptionkey@0"
I20260619 02:44:44.563066 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "493967ece2f44162ab9951bc96b514f6"
format_stamp: "Formatted at 2026-06-19 02:44:44 on dist-test-slave-pdnb"
server_key: "73e5c8be8e0e64b7ae9de039b5f9c6ff"
server_key_iv: "fc545718d077834a1343d5323b23a69e"
server_key_version: "encryptionkey@0"
I20260619 02:44:44.564428 28379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:44.565286 28394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:44.565492 28379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 02:44:44.565591 28379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "493967ece2f44162ab9951bc96b514f6"
format_stamp: "Formatted at 2026-06-19 02:44:44 on dist-test-slave-pdnb"
server_key: "73e5c8be8e0e64b7ae9de039b5f9c6ff"
server_key_iv: "fc545718d077834a1343d5323b23a69e"
server_key_version: "encryptionkey@0"
I20260619 02:44:44.565671 28379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:44.571797 28379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:44.573737 28379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:44.573892 28379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:44.577764 28379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40645
I20260619 02:44:44.577805 28446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40645 every 8 connection(s)
I20260619 02:44:44.578150 28379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:44.579118 28447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:44.581640 28447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: Bootstrap starting.
I20260619 02:44:44.582151 28447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:44.582383 28447 log.cc:824] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:44.583039 28447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: No bootstrap required, opened a new log
I20260619 02:44:44.583809 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28379
I20260619 02:44:44.583946 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:44.584124 7716 external_mini_cluster.cc:1428] Setting key 59cfe294a4244e9d84b7ca139fd3ecd5
I20260619 02:44:44.585094 28447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } }
I20260619 02:44:44.585268 28447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:44.585309 28447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493967ece2f44162ab9951bc96b514f6, State: Initialized, Role: FOLLOWER
I20260619 02:44:44.585480 28447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } }
I20260619 02:44:44.585569 28447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:44.585610 28447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:44.585660 28447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:44.586371 28447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } }
I20260619 02:44:44.586479 28447 leader_election.cc:302] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 493967ece2f44162ab9951bc96b514f6; no voters:
I20260619 02:44:44.586627 28447 leader_election.cc:288] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:44.586679 28452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:44.586839 28452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [term 1 LEADER]: Becoming Leader. State: Replica: 493967ece2f44162ab9951bc96b514f6, State: Running, Role: LEADER
I20260619 02:44:44.586848 28447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:44.587014 28452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } }
I20260619 02:44:44.588092 28453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "493967ece2f44162ab9951bc96b514f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } } }
I20260619 02:44:44.588179 28453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:44.588335 28454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 493967ece2f44162ab9951bc96b514f6. Latest consensus state: current_term: 1 leader_uuid: "493967ece2f44162ab9951bc96b514f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493967ece2f44162ab9951bc96b514f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40645 } } }
I20260619 02:44:44.588528 28454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6 [sys.catalog]: This master's current role is: LEADER
W20260619 02:44:44.588792 28467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:44.588845 28467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:44.588899 28468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:44.589052 28468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:44.590548 28468 catalog_manager.cc:1347] Generated new cluster ID: f799318d03754f569b2d425c2073c8df
I20260619 02:44:44.590607 28468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:44.590791 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:44.584642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35019 (local address 127.0.0.1:40645)
0619 02:44:44.584846 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:44.584856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:44.584892 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:44:44.585056 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:44.585063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:44.585121 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:44.585266 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:44.585510 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:44.587190 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:44.588461 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:44.589067 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:44.589225 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:44.589914 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:44.589918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:44.589918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:44.590170 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:44.590205 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:44.590220 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:44.590310 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:44.590385 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:44:44.590538 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260619 02:44:44.593170 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:44.599850 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:44.597063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56395 (local address 127.0.0.1:40645)
0619 02:44:44.597101 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:44.597102 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:44.597244 (+ 142us) server_negotiation.cc:407] Connection header received
0619 02:44:44.597316 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:44.597317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:44.597327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:44.597374 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:44.597499 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:44.597947 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:44.598656 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:44.598856 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:44.598936 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:44.599598 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:44.599600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:44.599600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:44.599604 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:44.599617 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:44.599621 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:44.599675 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:44.599733 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:44:44.599754 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260619 02:44:44.604995 28468 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:44.605494 28468 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:44.614424 28468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 493967ece2f44162ab9951bc96b514f6: Generated new TSK 0
I20260619 02:44:44.614620 28468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:44.625377 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28379
2026-06-19T02:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:44 dist-test-slave-pdnb krb5kdc[28478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:44 dist-test-slave-pdnb krb5kdc[28478](info): set up 2 sockets
Jun 19 02:44:44 dist-test-slave-pdnb krb5kdc[28478](info): commencing operation
krb5kdc: starting...
W20260619 02:44:45.660892 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:45 dist-test-slave-pdnb krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:45.777856 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:45.857187 28491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:45.857412 28491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:45.857450 28491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:45.858592 28491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:45.858659 28491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:45.858685 28491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:45.858714 28491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:45.858742 28491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:45.860280 28491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:45.860548 28491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:45.860788 28491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:45.865280 28499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:45.865309 28496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:45.865309 28497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:45.865588 28491 server_base.cc:1034] running on GCE node
I20260619 02:44:45.865811 28491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:45.866092 28491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:45.867264 28491 hybrid_clock.cc:648] HybridClock initialized: now 1781837085867257 us; error 35 us; skew 500 ppm
Jun 19 02:44:45 dist-test-slave-pdnb krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:45.874301 28491 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:45.874809 28491 webserver.cc:460] Webserver started at http://127.0.0.1:38703/ using document root <none> and password file <none>
I20260619 02:44:45.875056 28491 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:45.875128 28491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:45.875293 28491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:45.876353 28491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "eb13a7b055824862a655aeb1e442f2e7"
format_stamp: "Formatted at 2026-06-19 02:44:45 on dist-test-slave-pdnb"
server_key: "0794483a179daa44b6e99222c5b70a52"
server_key_iv: "f467ae24fbdfc755a9bd3ff8e84d826c"
server_key_version: "encryptionkey@0"
I20260619 02:44:45.876762 28491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "eb13a7b055824862a655aeb1e442f2e7"
format_stamp: "Formatted at 2026-06-19 02:44:45 on dist-test-slave-pdnb"
server_key: "0794483a179daa44b6e99222c5b70a52"
server_key_iv: "f467ae24fbdfc755a9bd3ff8e84d826c"
server_key_version: "encryptionkey@0"
I20260619 02:44:45.878166 28491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:45.878911 28506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:45.879078 28491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:45.879176 28491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "eb13a7b055824862a655aeb1e442f2e7"
format_stamp: "Formatted at 2026-06-19 02:44:45 on dist-test-slave-pdnb"
server_key: "0794483a179daa44b6e99222c5b70a52"
server_key_iv: "f467ae24fbdfc755a9bd3ff8e84d826c"
server_key_version: "encryptionkey@0"
I20260619 02:44:45.879262 28491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:45.882542 28491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:45.884308 28491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:45.884462 28491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:45.888365 28491 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40121
I20260619 02:44:45.888402 28558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40121 every 8 connection(s)
I20260619 02:44:45.888783 28491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:45.889683 28559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:45.891783 28559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7: Bootstrap starting.
I20260619 02:44:45.892365 28559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:45.892349 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28491
I20260619 02:44:45.892465 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:45.892663 7716 external_mini_cluster.cc:1428] Setting key 2dbe62103db7806e9cc3b808ef9d2078
I20260619 02:44:45.892692 28559 log.cc:824] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:45.893522 28559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7: No bootstrap required, opened a new log
I20260619 02:44:45.895326 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:45.893181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58593 (local address 127.0.0.1:40121)
0619 02:44:45.893350 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:45.893357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:45.893551 (+ 194us) server_negotiation.cc:407] Connection header received
0619 02:44:45.894029 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:45.894036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:45.894062 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:45.894224 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:45.894895 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:45.894898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:45.894898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:45.894902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:45.894934 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:45.894943 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:45.895012 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:45.895086 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 02:44:45.895145 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260619 02:44:45.896222 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:45.896179 28559 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } }
I20260619 02:44:45.896363 28559 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:45.896409 28559 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb13a7b055824862a655aeb1e442f2e7, State: Initialized, Role: FOLLOWER
I20260619 02:44:45.896518 28559 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } }
I20260619 02:44:45.896596 28559 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:45.896622 28559 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:45.896667 28559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:45.897403 28559 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } }
I20260619 02:44:45.897526 28559 leader_election.cc:302] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb13a7b055824862a655aeb1e442f2e7; no voters:
I20260619 02:44:45.897679 28559 leader_election.cc:288] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:45.897701 28568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:45.897809 28568 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [term 1 LEADER]: Becoming Leader. State: Replica: eb13a7b055824862a655aeb1e442f2e7, State: Running, Role: LEADER
I20260619 02:44:45.897914 28559 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:45.897892 28568 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } }
I20260619 02:44:45.898350 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb13a7b055824862a655aeb1e442f2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } } }
I20260619 02:44:45.898350 28570 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb13a7b055824862a655aeb1e442f2e7. Latest consensus state: current_term: 1 leader_uuid: "eb13a7b055824862a655aeb1e442f2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb13a7b055824862a655aeb1e442f2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40121 } } }
I20260619 02:44:45.898617 28570 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:45.898536 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:45.899032 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:45.897773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51025 (local address 127.0.0.1:40121)
0619 02:44:45.897813 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:45.897815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:45.898004 (+ 189us) server_negotiation.cc:407] Connection header received
0619 02:44:45.898113 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:45.898114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:45.898117 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:45.898141 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:45.898772 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:45.898774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:45.898774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:45.898778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:45.898790 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:45.898794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:45.898824 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:45.898936 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:44:45.898949 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:44:45.898962 28578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:45.899684 28578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:45.901122 28578 catalog_manager.cc:1347] Generated new cluster ID: afb433fbd2ac4baaa95e87be87bc0c52
I20260619 02:44:45.901177 28578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:45.911942 28578 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:45.912091 28578 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:45.922835 28578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb13a7b055824862a655aeb1e442f2e7: Generated new TSK 0
I20260619 02:44:45.923022 28578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:45.925081 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28491
2026-06-19T02:44:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:45 dist-test-slave-pdnb krb5kdc[28590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:45 dist-test-slave-pdnb krb5kdc[28590](info): set up 2 sockets
Jun 19 02:44:45 dist-test-slave-pdnb krb5kdc[28590](info): commencing operation
krb5kdc: starting...
W20260619 02:44:46.956790 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:47.075718 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:47.156616 28603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:47.156878 28603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:47.156917 28603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:47.158072 28603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:47.158139 28603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:47.158164 28603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:47.158196 28603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:47.158226 28603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:47.159659 28603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:47.159925 28603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:47.160138 28603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:47.164911 28609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:47.164942 28608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:47.164978 28603 server_base.cc:1034] running on GCE node
W20260619 02:44:47.164909 28611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:47.165334 28603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:47.165669 28603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:47.166841 28603 hybrid_clock.cc:648] HybridClock initialized: now 1781837087166822 us; error 36 us; skew 500 ppm
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:47.169265 28603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:47.169800 28603 webserver.cc:460] Webserver started at http://127.0.0.1:34021/ using document root <none> and password file <none>
I20260619 02:44:47.170053 28603 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:47.170122 28603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:47.170256 28603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:47.171285 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "7263b10954484c139a99838a3dfaaa97"
format_stamp: "Formatted at 2026-06-19 02:44:47 on dist-test-slave-pdnb"
server_key: "f92e01f05d81f72d84729828f6481f40"
server_key_iv: "b121c35a1c84cf660a96f944368f5b0a"
server_key_version: "encryptionkey@0"
I20260619 02:44:47.171669 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "7263b10954484c139a99838a3dfaaa97"
format_stamp: "Formatted at 2026-06-19 02:44:47 on dist-test-slave-pdnb"
server_key: "f92e01f05d81f72d84729828f6481f40"
server_key_iv: "b121c35a1c84cf660a96f944368f5b0a"
server_key_version: "encryptionkey@0"
I20260619 02:44:47.173003 28603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:47.173717 28618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:47.173888 28603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:47.173985 28603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "7263b10954484c139a99838a3dfaaa97"
format_stamp: "Formatted at 2026-06-19 02:44:47 on dist-test-slave-pdnb"
server_key: "f92e01f05d81f72d84729828f6481f40"
server_key_iv: "b121c35a1c84cf660a96f944368f5b0a"
server_key_version: "encryptionkey@0"
I20260619 02:44:47.174064 28603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:47.194521 28603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:47.196286 28603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:47.196456 28603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:47.200393 28603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41839
I20260619 02:44:47.200428 28670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41839 every 8 connection(s)
I20260619 02:44:47.200788 28603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:47.201712 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:47.203850 28671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97: Bootstrap starting.
I20260619 02:44:47.204368 28671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:47.204594 28671 log.cc:824] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:47.205237 28671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97: No bootstrap required, opened a new log
I20260619 02:44:47.206709 28671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } }
I20260619 02:44:47.206841 28671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:47.206863 28671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7263b10954484c139a99838a3dfaaa97, State: Initialized, Role: FOLLOWER
I20260619 02:44:47.206959 28671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } }
I20260619 02:44:47.207010 28671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:47.207031 28671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:47.207108 28671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:47.207616 28671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } }
I20260619 02:44:47.207728 28671 leader_election.cc:302] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7263b10954484c139a99838a3dfaaa97; no voters:
I20260619 02:44:47.207911 28671 leader_election.cc:288] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:47.207967 28674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:47.208117 28674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [term 1 LEADER]: Becoming Leader. State: Replica: 7263b10954484c139a99838a3dfaaa97, State: Running, Role: LEADER
I20260619 02:44:47.208158 28671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:47.208254 28674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } }
I20260619 02:44:47.208678 28676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7263b10954484c139a99838a3dfaaa97. Latest consensus state: current_term: 1 leader_uuid: "7263b10954484c139a99838a3dfaaa97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } } }
I20260619 02:44:47.208808 28675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7263b10954484c139a99838a3dfaaa97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7263b10954484c139a99838a3dfaaa97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41839 } } }
I20260619 02:44:47.208913 28675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:47.208830 28676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:47.209163 28683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:47.209744 28683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:47.210827 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28603
I20260619 02:44:47.210959 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:47.211129 7716 external_mini_cluster.cc:1428] Setting key d3042bda77abdd07ae58b202dc62356a
I20260619 02:44:47.213129 28683 catalog_manager.cc:1347] Generated new cluster ID: 82441e187bcd4bef8891087138a1029c
I20260619 02:44:47.213186 28683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:44:47.221943 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:47.211588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:48189 (local address 127.0.0.1:41839)
0619 02:44:47.213163 (+ 1575us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:47.213170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:47.213222 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:44:47.213348 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:47.213353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:47.213404 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:47.213537 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:47.214152 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:47.215063 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:47.215804 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:47.216211 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:47.216325 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:47.218640 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:47.218648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:47.218649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:47.218659 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:47.220299 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:47.220681 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:47.220684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:47.220685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:47.220819 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:47.221054 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:47.221055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:47.221056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:47.221275 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:47.221447 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:47.221567 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 02:44:47.221770 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":69,"threads_started":1}
I20260619 02:44:47.222822 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:47.230100 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:47.224121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:48297 (local address 127.0.0.1:41839)
0619 02:44:47.224201 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:47.224203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:47.224251 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:44:47.224321 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:47.224322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:47.224331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:47.224385 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:47.224948 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:47.225429 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:47.226120 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:47.226331 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:47.226415 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:47.227531 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:47.227532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:47.227532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:47.227539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:47.228982 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:47.229339 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:47.229340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:47.229341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:47.229398 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:47.229616 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:47.229617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:47.229618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:47.229720 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:47.229847 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:47.229950 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:44:47.230009 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:44:47.231323 28683 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:47.232003 28683 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:47.237726 28683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7263b10954484c139a99838a3dfaaa97: Generated new TSK 0
I20260619 02:44:47.237928 28683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:47.242851 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28603
W20260619 02:44:47.246655 28588 connection.cc:535] client connection to 127.0.0.1:41839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41839): Connection reset by peer (error 104)
2026-06-19T02:44:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28702](info): set up 2 sockets
Jun 19 02:44:47 dist-test-slave-pdnb krb5kdc[28702](info): commencing operation
krb5kdc: starting...
W20260619 02:44:48.280771 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:48 dist-test-slave-pdnb krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:48.394986 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:48.473793 28715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:48.474028 28715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:48.474066 28715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:48.475193 28715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:48.475262 28715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:48.475287 28715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:48.475314 28715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:48.475353 28715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:48.476845 28715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:48.477121 28715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:48.477351 28715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:48.481904 28723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:48.481984 28715 server_base.cc:1034] running on GCE node
W20260619 02:44:48.481922 28720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:48.481920 28721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:48.482424 28715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:48.482652 28715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:48.483798 28715 hybrid_clock.cc:648] HybridClock initialized: now 1781837088483791 us; error 30 us; skew 500 ppm
Jun 19 02:44:48 dist-test-slave-pdnb krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837088, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:48.486205 28715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:48.486563 28715 webserver.cc:460] Webserver started at http://127.0.0.1:38387/ using document root <none> and password file <none>
I20260619 02:44:48.486759 28715 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:48.486797 28715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:48.486884 28715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:48.487843 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "a5de012fb1c7480f8918789075ea46d7"
format_stamp: "Formatted at 2026-06-19 02:44:48 on dist-test-slave-pdnb"
server_key: "9b9d0eb4ade4313f98c793d4f7d4d81b"
server_key_iv: "573e47a4c1183e5f703aa9f716627a4f"
server_key_version: "encryptionkey@0"
I20260619 02:44:48.488124 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "a5de012fb1c7480f8918789075ea46d7"
format_stamp: "Formatted at 2026-06-19 02:44:48 on dist-test-slave-pdnb"
server_key: "9b9d0eb4ade4313f98c793d4f7d4d81b"
server_key_iv: "573e47a4c1183e5f703aa9f716627a4f"
server_key_version: "encryptionkey@0"
I20260619 02:44:48.489409 28715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:44:48.490041 28730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:48.490226 28715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:48.490306 28715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "a5de012fb1c7480f8918789075ea46d7"
format_stamp: "Formatted at 2026-06-19 02:44:48 on dist-test-slave-pdnb"
server_key: "9b9d0eb4ade4313f98c793d4f7d4d81b"
server_key_iv: "573e47a4c1183e5f703aa9f716627a4f"
server_key_version: "encryptionkey@0"
I20260619 02:44:48.490360 28715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:48.506178 28715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:48.507910 28715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:48.508014 28715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:48.511739 28715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40731
I20260619 02:44:48.511791 28782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40731 every 8 connection(s)
I20260619 02:44:48.512097 28715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:48.513145 28783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:48.515275 28783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7: Bootstrap starting.
I20260619 02:44:48.515825 28783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:48.516108 28783 log.cc:824] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:48.516829 28783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7: No bootstrap required, opened a new log
I20260619 02:44:48.518344 28783 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } }
I20260619 02:44:48.518496 28783 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:48.518541 28783 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5de012fb1c7480f8918789075ea46d7, State: Initialized, Role: FOLLOWER
I20260619 02:44:48.518698 28783 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } }
I20260619 02:44:48.518795 28783 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:48.518842 28783 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:48.518894 28783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:48.519423 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28715
I20260619 02:44:48.519456 28783 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } }
I20260619 02:44:48.519532 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:48.519557 28783 leader_election.cc:302] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5de012fb1c7480f8918789075ea46d7; no voters:
I20260619 02:44:48.519719 28783 leader_election.cc:288] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:48.519726 7716 external_mini_cluster.cc:1428] Setting key b1b7249e87ce1b15b2edb9feddfef231
I20260619 02:44:48.519767 28786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:48.519956 28786 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [term 1 LEADER]: Becoming Leader. State: Replica: a5de012fb1c7480f8918789075ea46d7, State: Running, Role: LEADER
I20260619 02:44:48.520094 28786 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } }
I20260619 02:44:48.519976 28783 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:48.520505 28788 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5de012fb1c7480f8918789075ea46d7. Latest consensus state: current_term: 1 leader_uuid: "a5de012fb1c7480f8918789075ea46d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } } }
I20260619 02:44:48.520517 28787 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5de012fb1c7480f8918789075ea46d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5de012fb1c7480f8918789075ea46d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40731 } } }
I20260619 02:44:48.520596 28788 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:48.520624 28787 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:48.520785 28792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:48.521361 28792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:48.522965 28792 catalog_manager.cc:1347] Generated new cluster ID: 0d38bd7cf53c4cae9f2b5f158dcecdcd
I20260619 02:44:48.523015 28792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:48.526437 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:48.520140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54341 (local address 127.0.0.1:40731)
0619 02:44:48.520376 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:48.520388 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:48.521072 (+ 684us) server_negotiation.cc:407] Connection header received
0619 02:44:48.521189 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:48.521196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:48.521258 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:48.521396 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:48.521615 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:48.523100 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:48.524163 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:48.524802 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:48.524912 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:48.525599 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:48.525603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:48.525603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:48.525867 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:48.525907 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:48.525921 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:48.526006 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:48.526200 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 02:44:48.526230 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260619 02:44:48.527339 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:48.532279 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:48.529018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:41947 (local address 127.0.0.1:40731)
0619 02:44:48.529080 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:48.529082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:48.529115 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:44:48.529168 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:48.529168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:48.529188 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:48.529256 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:48.529375 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:48.530084 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:48.531041 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:48.531357 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:48.531432 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:48.532037 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:48.532038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:48.532038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:48.532042 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:48.532055 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:48.532059 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:48.532123 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:48.532194 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 02:44:48.532204 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:44:48.533741 28792 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:48.534413 28792 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:48.541414 28792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5de012fb1c7480f8918789075ea46d7: Generated new TSK 0
I20260619 02:44:48.541602 28792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:48.549774 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28715
W20260619 02:44:48.553745 28700 connection.cc:535] client connection to 127.0.0.1:40731 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40731): Connection reset by peer (error 104)
2026-06-19T02:44:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:48 dist-test-slave-pdnb krb5kdc[28814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:48 dist-test-slave-pdnb krb5kdc[28814](info): set up 2 sockets
Jun 19 02:44:48 dist-test-slave-pdnb krb5kdc[28814](info): commencing operation
krb5kdc: starting...
W20260619 02:44:49.584884 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:49 dist-test-slave-pdnb krb5kdc[28814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:49.699067 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:49.779094 28827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:49.779337 28827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:49.779361 28827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:49.780493 28827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:49.780532 28827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:49.780547 28827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:49.780596 28827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:49.780616 28827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:49.782133 28827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:49.782384 28827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:49.782609 28827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:49.787386 28835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:49.787396 28833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:49.787372 28832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:49.787577 28827 server_base.cc:1034] running on GCE node
I20260619 02:44:49.788053 28827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:49.788348 28827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:49.789505 28827 hybrid_clock.cc:648] HybridClock initialized: now 1781837089789501 us; error 36 us; skew 500 ppm
Jun 19 02:44:49 dist-test-slave-pdnb krb5kdc[28814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:49.791769 28827 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:49.792161 28827 webserver.cc:460] Webserver started at http://127.0.0.1:36113/ using document root <none> and password file <none>
I20260619 02:44:49.792398 28827 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:49.792466 28827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:49.792611 28827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:49.793664 28827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "2b0867e001b3411db5181d11354303ab"
format_stamp: "Formatted at 2026-06-19 02:44:49 on dist-test-slave-pdnb"
server_key: "cc559a6d51053a8902d598cd969a1c0f"
server_key_iv: "2e63087fb2ab9f976698bf54268083b6"
server_key_version: "encryptionkey@0"
I20260619 02:44:49.794006 28827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "2b0867e001b3411db5181d11354303ab"
format_stamp: "Formatted at 2026-06-19 02:44:49 on dist-test-slave-pdnb"
server_key: "cc559a6d51053a8902d598cd969a1c0f"
server_key_iv: "2e63087fb2ab9f976698bf54268083b6"
server_key_version: "encryptionkey@0"
I20260619 02:44:49.795261 28827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:44:49.796000 28842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:49.796169 28827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:49.796263 28827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "2b0867e001b3411db5181d11354303ab"
format_stamp: "Formatted at 2026-06-19 02:44:49 on dist-test-slave-pdnb"
server_key: "cc559a6d51053a8902d598cd969a1c0f"
server_key_iv: "2e63087fb2ab9f976698bf54268083b6"
server_key_version: "encryptionkey@0"
I20260619 02:44:49.796356 28827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:49.799532 28827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:49.801302 28827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:49.801448 28827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:49.805227 28827 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33109
I20260619 02:44:49.805255 28894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33109 every 8 connection(s)
I20260619 02:44:49.805564 28827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:49.806481 28895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:49.808537 28895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab: Bootstrap starting.
I20260619 02:44:49.809105 28895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:49.809335 28895 log.cc:824] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:49.809939 28895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab: No bootstrap required, opened a new log
I20260619 02:44:49.811426 28895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } }
I20260619 02:44:49.811553 28895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:49.811574 28895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b0867e001b3411db5181d11354303ab, State: Initialized, Role: FOLLOWER
I20260619 02:44:49.811715 28895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } }
I20260619 02:44:49.811818 28895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:49.811842 28895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:49.811944 28895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:49.812466 28895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } }
I20260619 02:44:49.812574 28895 leader_election.cc:302] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b0867e001b3411db5181d11354303ab; no voters:
I20260619 02:44:49.812768 28895 leader_election.cc:288] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:49.812808 28898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:49.812963 28898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [term 1 LEADER]: Becoming Leader. State: Replica: 2b0867e001b3411db5181d11354303ab, State: Running, Role: LEADER
I20260619 02:44:49.813026 28895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:49.813086 28898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } }
I20260619 02:44:49.813443 28900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b0867e001b3411db5181d11354303ab. Latest consensus state: current_term: 1 leader_uuid: "2b0867e001b3411db5181d11354303ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } } }
I20260619 02:44:49.813536 28900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:49.813556 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28827
I20260619 02:44:49.813572 28899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b0867e001b3411db5181d11354303ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0867e001b3411db5181d11354303ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33109 } } }
I20260619 02:44:49.813647 28899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:49.813657 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:49.813835 7716 external_mini_cluster.cc:1428] Setting key e67fb0477b2f10a328ffb2e7bcb03625
I20260619 02:44:49.814399 28906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:49.815011 28906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:49.816182 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:49.814472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35511 (local address 127.0.0.1:33109)
0619 02:44:49.814637 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:49.814646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:49.814683 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:44:49.814784 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:49.814788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:49.814816 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:49.814977 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:49.815632 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:49.815635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:49.815636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:49.815641 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:49.815690 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:49.815705 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:49.815760 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:49.815966 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 02:44:49.816008 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260619 02:44:49.816738 28906 catalog_manager.cc:1347] Generated new cluster ID: ae618ef6273c4aa2afe836875dddadfc
I20260619 02:44:49.816797 28906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:49.821313 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:49.823836 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:49.822711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:34257 (local address 127.0.0.1:33109)
0619 02:44:49.822740 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:49.822742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:49.822894 (+ 152us) server_negotiation.cc:407] Connection header received
0619 02:44:49.822951 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:49.822952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:49.822956 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:49.823012 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:49.823604 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:49.823605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:49.823605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:49.823609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:49.823624 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:49.823628 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:49.823685 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:49.823763 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:44:49.823775 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:44:49.825028 28906 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:49.825169 28906 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:49.836016 28906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b0867e001b3411db5181d11354303ab: Generated new TSK 0
I20260619 02:44:49.836191 28906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:49.839823 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28827
2026-06-19T02:44:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:49 dist-test-slave-pdnb krb5kdc[28926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:49 dist-test-slave-pdnb krb5kdc[28926](info): set up 2 sockets
Jun 19 02:44:49 dist-test-slave-pdnb krb5kdc[28926](info): commencing operation
krb5kdc: starting...
W20260619 02:44:50.872841 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:50 dist-test-slave-pdnb krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:50.986793 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:51.065747 28939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:51.065958 28939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:51.065981 28939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:51.067067 28939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:51.067102 28939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:51.067117 28939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:51.067129 28939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:51.067142 28939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:51.068509 28939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:51.068689 28939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:51.068946 28939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:51.073832 28947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:51.074004 28939 server_base.cc:1034] running on GCE node
W20260619 02:44:51.073832 28945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:51.073932 28944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:51.074327 28939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:51.074615 28939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:51.075789 28939 hybrid_clock.cc:648] HybridClock initialized: now 1781837091075776 us; error 30 us; skew 500 ppm
Jun 19 02:44:51 dist-test-slave-pdnb krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837091, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:51.082476 28939 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:51.082950 28939 webserver.cc:460] Webserver started at http://127.0.0.1:34381/ using document root <none> and password file <none>
I20260619 02:44:51.083190 28939 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:51.083266 28939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:51.083408 28939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:51.084482 28939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "5e393f72594a4ed1b77188bde576b9df"
format_stamp: "Formatted at 2026-06-19 02:44:51 on dist-test-slave-pdnb"
server_key: "6be241cedd0d8c975aa481727248e6fe"
server_key_iv: "c61f289b6cd2bf1d83db9353dbf8de95"
server_key_version: "encryptionkey@0"
I20260619 02:44:51.084872 28939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "5e393f72594a4ed1b77188bde576b9df"
format_stamp: "Formatted at 2026-06-19 02:44:51 on dist-test-slave-pdnb"
server_key: "6be241cedd0d8c975aa481727248e6fe"
server_key_iv: "c61f289b6cd2bf1d83db9353dbf8de95"
server_key_version: "encryptionkey@0"
I20260619 02:44:51.086170 28939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:44:51.086885 28954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:51.087071 28939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:51.087167 28939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "5e393f72594a4ed1b77188bde576b9df"
format_stamp: "Formatted at 2026-06-19 02:44:51 on dist-test-slave-pdnb"
server_key: "6be241cedd0d8c975aa481727248e6fe"
server_key_iv: "c61f289b6cd2bf1d83db9353dbf8de95"
server_key_version: "encryptionkey@0"
I20260619 02:44:51.087246 28939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:51.100693 28939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:51.102890 28939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:51.103039 28939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:51.106909 28939 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36383
I20260619 02:44:51.106938 29006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36383 every 8 connection(s)
I20260619 02:44:51.107276 28939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:51.108234 29007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:51.110503 29007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df: Bootstrap starting.
I20260619 02:44:51.111004 29007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:51.111222 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 28939
I20260619 02:44:51.111304 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:51.111243 29007 log.cc:824] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:51.111506 7716 external_mini_cluster.cc:1428] Setting key 41c86be4f727a6bd708eab585862ccd4
I20260619 02:44:51.112088 29007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df: No bootstrap required, opened a new log
I20260619 02:44:51.114751 29007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260619 02:44:51.114920 29007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:51.114951 29007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e393f72594a4ed1b77188bde576b9df, State: Initialized, Role: FOLLOWER
I20260619 02:44:51.115090 29007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260619 02:44:51.115195 29007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:51.115239 29007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:51.115288 29007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:51.115839 29007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260619 02:44:51.115937 29007 leader_election.cc:302] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e393f72594a4ed1b77188bde576b9df; no voters:
I20260619 02:44:51.116086 29007 leader_election.cc:288] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:51.116148 29012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:51.116314 29007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:51.116350 29012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [term 1 LEADER]: Becoming Leader. State: Replica: 5e393f72594a4ed1b77188bde576b9df, State: Running, Role: LEADER
I20260619 02:44:51.116520 29012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260619 02:44:51.116911 29013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e393f72594a4ed1b77188bde576b9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260619 02:44:51.116992 29013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:51.117522 29014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e393f72594a4ed1b77188bde576b9df. Latest consensus state: current_term: 1 leader_uuid: "5e393f72594a4ed1b77188bde576b9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e393f72594a4ed1b77188bde576b9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260619 02:44:51.117646 29014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:51.117889 29021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:51.118392 29021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 02:44:51 dist-test-slave-pdnb krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:44:51.119817 29021 catalog_manager.cc:1347] Generated new cluster ID: 24c74c8487334cf781ba8de7a4022adf
I20260619 02:44:51.119865 29021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:51.123606 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:51.112000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:46593 (local address 127.0.0.1:36383)
0619 02:44:51.112187 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:51.112196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:51.112237 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:44:51.112715 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:51.112723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:51.112807 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:51.112947 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:51.113523 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:51.114970 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:51.115790 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:51.116418 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:51.116876 (+ 458us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:51.119342 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:51.119345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:51.119346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:51.119357 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:51.121996 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:51.122394 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:51.122397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:51.122397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:51.122484 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:51.122717 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:51.122718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:51.122719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:51.122944 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:51.123136 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:51.123229 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:44:51.123452 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260619 02:44:51.124804 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:51.132215 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:51.126160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:52523 (local address 127.0.0.1:36383)
0619 02:44:51.126198 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:51.126200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:51.126331 (+ 131us) server_negotiation.cc:407] Connection header received
0619 02:44:51.126402 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:51.126403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:51.126413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:51.126467 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:51.126972 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:51.127465 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:51.128156 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:51.128386 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:51.128464 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:51.129659 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:51.129661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:51.129661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:51.129668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:51.131055 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:51.131419 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:51.131420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:51.131421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:51.131472 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:51.131696 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:51.131697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:51.131697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:51.131812 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:51.131920 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:51.132040 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 02:44:51.132101 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:51.146579 29021 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:51.147106 29021 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:51.151883 29021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e393f72594a4ed1b77188bde576b9df: Generated new TSK 0
I20260619 02:44:51.152060 29021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:51.157467 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 28939
W20260619 02:44:51.161731 28924 connection.cc:535] client connection to 127.0.0.1:36383 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36383): Connection reset by peer (error 104)
2026-06-19T02:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:51 dist-test-slave-pdnb krb5kdc[29038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:51 dist-test-slave-pdnb krb5kdc[29038](info): set up 2 sockets
Jun 19 02:44:51 dist-test-slave-pdnb krb5kdc[29038](info): commencing operation
krb5kdc: starting...
W20260619 02:44:52.185006 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:52 dist-test-slave-pdnb krb5kdc[29038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:52.296802 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:52.376010 29051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:52.376212 29051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:52.376241 29051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:52.377367 29051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:52.377403 29051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:52.377418 29051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:52.377431 29051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:52.377444 29051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:52.378823 29051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:52.379005 29051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:52.379166 29051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:52.383803 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:52.383802 29056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:52.383823 29057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:52.383994 29051 server_base.cc:1034] running on GCE node
I20260619 02:44:52.384342 29051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:52.384649 29051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:52.385802 29051 hybrid_clock.cc:648] HybridClock initialized: now 1781837092385804 us; error 44 us; skew 500 ppm
Jun 19 02:44:52 dist-test-slave-pdnb krb5kdc[29038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:52.388223 29051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:52.388685 29051 webserver.cc:460] Webserver started at http://127.0.0.1:44023/ using document root <none> and password file <none>
I20260619 02:44:52.388928 29051 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:52.388973 29051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:52.389060 29051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:52.390035 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "5380dcbe97034bf5baa63b5e5b8601d0"
format_stamp: "Formatted at 2026-06-19 02:44:52 on dist-test-slave-pdnb"
server_key: "95f8404773d5ad1f04f9ac3404df0a7e"
server_key_iv: "7442c5deaf73521a3945688f5715e9e9"
server_key_version: "encryptionkey@0"
I20260619 02:44:52.390331 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "5380dcbe97034bf5baa63b5e5b8601d0"
format_stamp: "Formatted at 2026-06-19 02:44:52 on dist-test-slave-pdnb"
server_key: "95f8404773d5ad1f04f9ac3404df0a7e"
server_key_iv: "7442c5deaf73521a3945688f5715e9e9"
server_key_version: "encryptionkey@0"
I20260619 02:44:52.391497 29051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:44:52.392133 29066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:52.392308 29051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:52.392410 29051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "5380dcbe97034bf5baa63b5e5b8601d0"
format_stamp: "Formatted at 2026-06-19 02:44:52 on dist-test-slave-pdnb"
server_key: "95f8404773d5ad1f04f9ac3404df0a7e"
server_key_iv: "7442c5deaf73521a3945688f5715e9e9"
server_key_version: "encryptionkey@0"
I20260619 02:44:52.392486 29051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:52.420284 29051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:52.422710 29051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:52.422886 29051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:52.426962 29051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43867
I20260619 02:44:52.427000 29118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43867 every 8 connection(s)
I20260619 02:44:52.427382 29051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:52.428215 29119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:52.430588 29119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0: Bootstrap starting.
I20260619 02:44:52.431197 29119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:52.431440 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29051
I20260619 02:44:52.431593 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:52.431577 29119 log.cc:824] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:52.431811 7716 external_mini_cluster.cc:1428] Setting key bfd26a6d59ff87352ed3861e2ef52054
I20260619 02:44:52.432421 29119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0: No bootstrap required, opened a new log
I20260619 02:44:52.435019 29119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260619 02:44:52.435190 29119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:52.435222 29119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5380dcbe97034bf5baa63b5e5b8601d0, State: Initialized, Role: FOLLOWER
I20260619 02:44:52.435485 29119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260619 02:44:52.435613 29119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:52.435712 29119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:52.435787 29119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:52.436501 29119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260619 02:44:52.436648 29119 leader_election.cc:302] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5380dcbe97034bf5baa63b5e5b8601d0; no voters:
I20260619 02:44:52.436861 29119 leader_election.cc:288] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:52.436930 29124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:52.437126 29124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [term 1 LEADER]: Becoming Leader. State: Replica: 5380dcbe97034bf5baa63b5e5b8601d0, State: Running, Role: LEADER
I20260619 02:44:52.437142 29119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:52.437250 29124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260619 02:44:52.437578 29124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5380dcbe97034bf5baa63b5e5b8601d0. Latest consensus state: current_term: 1 leader_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } } }
I20260619 02:44:52.437670 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5380dcbe97034bf5baa63b5e5b8601d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } } }
I20260619 02:44:52.437711 29124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:52.437731 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:52.437920 29131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:52.438109 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:52.432351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:38585 (local address 127.0.0.1:43867)
0619 02:44:52.432526 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:52.432536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:52.432571 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:44:52.433087 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:52.433094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:52.433148 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:52.433261 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:52.433481 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:52.434343 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:52.435641 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:52.436146 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:52.436225 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:52.437058 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:52.437061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:52.437062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:52.437264 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:52.437315 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:52.437333 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:52.437413 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:52.437822 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 02:44:52.437854 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260619 02:44:52.438907 29131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:52.439294 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:52.440675 29131 catalog_manager.cc:1347] Generated new cluster ID: 273f56b4bdb04cb68514b1160d3d5fec
I20260619 02:44:52.440758 29131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:52.448132 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:52.440891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:39831 (local address 127.0.0.1:43867)
0619 02:44:52.444804 (+ 3913us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:52.444806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:52.444813 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:44:52.444831 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:52.444832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:52.444842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:52.444891 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:52.445044 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:52.445766 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:52.446792 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:52.447095 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:52.447152 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:52.447863 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:52.447864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:52.447864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:52.447868 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:52.447882 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:52.447886 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:52.447933 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:52.448028 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 02:44:52.448038 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3889}
I20260619 02:44:52.459350 29131 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:52.459987 29131 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:52.464154 29131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5380dcbe97034bf5baa63b5e5b8601d0: Generated new TSK 0
I20260619 02:44:52.464354 29131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:52.467453 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29051
W20260619 02:44:52.471693 29036 connection.cc:535] client connection to 127.0.0.1:43867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43867): Connection reset by peer (error 104)
2026-06-19T02:44:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:52 dist-test-slave-pdnb krb5kdc[29150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:52 dist-test-slave-pdnb krb5kdc[29150](info): set up 2 sockets
Jun 19 02:44:52 dist-test-slave-pdnb krb5kdc[29150](info): commencing operation
krb5kdc: starting...
W20260619 02:44:53.512866 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:53 dist-test-slave-pdnb krb5kdc[29150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:44:53.624498 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:53.703022 29163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:53.703274 29163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:53.703313 29163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:53.704916 29163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:53.705021 29163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:53.705055 29163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:53.705089 29163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:53.705116 29163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:53.706872 29163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:53.707190 29163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:53.707432 29163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:53.712603 29169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:53.712603 29168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:53.712628 29171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:53.712752 29163 server_base.cc:1034] running on GCE node
I20260619 02:44:53.713032 29163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:53.713268 29163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:53.714406 29163 hybrid_clock.cc:648] HybridClock initialized: now 1781837093714389 us; error 42 us; skew 500 ppm
Jun 19 02:44:53 dist-test-slave-pdnb krb5kdc[29150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837093, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:53.716921 29163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:44:53.717378 29163 webserver.cc:460] Webserver started at http://127.0.0.1:43639/ using document root <none> and password file <none>
I20260619 02:44:53.717597 29163 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:53.717640 29163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:53.717785 29163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:53.718832 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "4cc6806c71eb425793e7cc5bca733dc0"
format_stamp: "Formatted at 2026-06-19 02:44:53 on dist-test-slave-pdnb"
server_key: "cf3e1277e33cb7ee3ea8b8cdaadc562d"
server_key_iv: "ae7ab6a9d086c5e6a1f07c452eedfe3a"
server_key_version: "encryptionkey@0"
I20260619 02:44:53.719177 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "4cc6806c71eb425793e7cc5bca733dc0"
format_stamp: "Formatted at 2026-06-19 02:44:53 on dist-test-slave-pdnb"
server_key: "cf3e1277e33cb7ee3ea8b8cdaadc562d"
server_key_iv: "ae7ab6a9d086c5e6a1f07c452eedfe3a"
server_key_version: "encryptionkey@0"
I20260619 02:44:53.720461 29163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 02:44:53.721241 29178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:53.721431 29163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:53.721515 29163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "4cc6806c71eb425793e7cc5bca733dc0"
format_stamp: "Formatted at 2026-06-19 02:44:53 on dist-test-slave-pdnb"
server_key: "cf3e1277e33cb7ee3ea8b8cdaadc562d"
server_key_iv: "ae7ab6a9d086c5e6a1f07c452eedfe3a"
server_key_version: "encryptionkey@0"
I20260619 02:44:53.721567 29163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:53.724579 29163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:53.726330 29163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:53.726428 29163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:53.729984 29163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41079
I20260619 02:44:53.730068 29230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41079 every 8 connection(s)
I20260619 02:44:53.730345 29163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:53.731283 29231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:53.733454 29231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0: Bootstrap starting.
I20260619 02:44:53.733960 29231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:53.734182 29231 log.cc:824] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:53.734797 29231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0: No bootstrap required, opened a new log
I20260619 02:44:53.736315 29231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } }
I20260619 02:44:53.736444 29231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:53.736500 29231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc6806c71eb425793e7cc5bca733dc0, State: Initialized, Role: FOLLOWER
I20260619 02:44:53.736632 29231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } }
I20260619 02:44:53.736708 29231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:53.736794 29231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:53.736830 29231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:53.737322 29231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } }
I20260619 02:44:53.737404 29231 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc6806c71eb425793e7cc5bca733dc0; no voters:
I20260619 02:44:53.737524 29231 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:53.737576 29234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:53.737730 29234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [term 1 LEADER]: Becoming Leader. State: Replica: 4cc6806c71eb425793e7cc5bca733dc0, State: Running, Role: LEADER
I20260619 02:44:53.737814 29231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:53.737857 29234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } }
I20260619 02:44:53.738266 29235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc6806c71eb425793e7cc5bca733dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } } }
I20260619 02:44:53.738370 29235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:53.738271 29236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc6806c71eb425793e7cc5bca733dc0. Latest consensus state: current_term: 1 leader_uuid: "4cc6806c71eb425793e7cc5bca733dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc6806c71eb425793e7cc5bca733dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41079 } } }
I20260619 02:44:53.738453 29236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:53.738636 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29163
I20260619 02:44:53.738757 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:53.738919 29238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:53.738937 7716 external_mini_cluster.cc:1428] Setting key e514385dc9169dc4148292e780f67c07
I20260619 02:44:53.739450 29238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:53.741184 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:53.739560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40765 (local address 127.0.0.1:41079)
0619 02:44:53.739705 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:53.739722 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:53.739753 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:44:53.739871 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:53.739878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:53.739908 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:53.740044 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:53.740686 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:53.740690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:53.740690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:53.740694 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:53.740789 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:53.740805 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:53.740866 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:53.740935 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:44:53.740978 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260619 02:44:53.741475 29238 catalog_manager.cc:1347] Generated new cluster ID: 568263d5187e4b3a84ffd9636d8dba26
I20260619 02:44:53.741526 29238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:53.742050 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:53.744369 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:53.743422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:38351 (local address 127.0.0.1:41079)
0619 02:44:53.743474 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:53.743476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:53.743592 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:44:53.743641 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:53.743641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:53.743645 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:53.743683 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:53.744151 (+ 468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:53.744152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:53.744152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:53.744155 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:53.744167 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:53.744170 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:53.744221 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:53.744264 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:44:53.744274 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 02:44:53.762132 29238 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:53.762311 29238 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:53.767503 29238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cc6806c71eb425793e7cc5bca733dc0: Generated new TSK 0
I20260619 02:44:53.767674 29238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:53.770367 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29163
2026-06-19T02:44:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:53 dist-test-slave-pdnb krb5kdc[29262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:53 dist-test-slave-pdnb krb5kdc[29262](info): set up 2 sockets
Jun 19 02:44:53 dist-test-slave-pdnb krb5kdc[29262](info): commencing operation
krb5kdc: starting...
W20260619 02:44:54.844887 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:54 dist-test-slave-pdnb krb5kdc[29262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:54.959956 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35265
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:35265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:55.040654 29275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:55.040926 29275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:55.040959 29275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:55.042059 29275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:55.042119 29275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:55.042153 29275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:55.042194 29275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:55.042225 29275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:55.043637 29275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:35265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:35265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:55.043902 29275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:55.044111 29275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:55.048720 29280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:55.048780 29281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:55.048934 29275 server_base.cc:1034] running on GCE node
W20260619 02:44:55.048945 29283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:55.049327 29275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:55.049618 29275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:55.050796 29275 hybrid_clock.cc:648] HybridClock initialized: now 1781837095050783 us; error 39 us; skew 500 ppm
Jun 19 02:44:55 dist-test-slave-pdnb krb5kdc[29262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837095, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:55.053411 29275 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:55.053836 29275 webserver.cc:460] Webserver started at http://127.7.137.62:35417/ using document root <none> and password file <none>
I20260619 02:44:55.054062 29275 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:55.054105 29275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:55.054193 29275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:55.055142 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "eb428d87d7534482a5d284c35365aeec"
format_stamp: "Formatted at 2026-06-19 02:44:55 on dist-test-slave-pdnb"
server_key: "243fdaac155f566733215140a4af578e"
server_key_iv: "ea357e9c88c79510e6d5e227de7289b9"
server_key_version: "encryptionkey@0"
I20260619 02:44:55.055438 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "eb428d87d7534482a5d284c35365aeec"
format_stamp: "Formatted at 2026-06-19 02:44:55 on dist-test-slave-pdnb"
server_key: "243fdaac155f566733215140a4af578e"
server_key_iv: "ea357e9c88c79510e6d5e227de7289b9"
server_key_version: "encryptionkey@0"
I20260619 02:44:55.056658 29275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:44:55.057425 29290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:55.057578 29275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 02:44:55.057674 29275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "eb428d87d7534482a5d284c35365aeec"
format_stamp: "Formatted at 2026-06-19 02:44:55 on dist-test-slave-pdnb"
server_key: "243fdaac155f566733215140a4af578e"
server_key_iv: "ea357e9c88c79510e6d5e227de7289b9"
server_key_version: "encryptionkey@0"
I20260619 02:44:55.057760 29275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:55.075423 29275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:55.077869 29275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:55.078034 29275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:55.082187 29275 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:35265
I20260619 02:44:55.082232 29342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:35265 every 8 connection(s)
I20260619 02:44:55.082584 29275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:55.083418 29343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:55.084472 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29275
I20260619 02:44:55.084614 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:55.084834 7716 external_mini_cluster.cc:1428] Setting key 0e15f0863f757c4d190b7b6a8e857da4
I20260619 02:44:55.086249 29343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec: Bootstrap starting.
I20260619 02:44:55.086833 29343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:55.087122 29343 log.cc:824] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:55.087831 29343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec: No bootstrap required, opened a new log
I20260619 02:44:55.089747 29343 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } }
I20260619 02:44:55.089900 29343 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:55.089946 29343 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb428d87d7534482a5d284c35365aeec, State: Initialized, Role: FOLLOWER
I20260619 02:44:55.090091 29343 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } }
I20260619 02:44:55.090188 29343 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:55.090231 29343 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:55.090278 29343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:55.090802 29343 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } }
I20260619 02:44:55.090914 29343 leader_election.cc:302] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb428d87d7534482a5d284c35365aeec; no voters:
I20260619 02:44:55.091099 29343 leader_election.cc:288] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:55.091120 29348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:55.091225 29348 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [term 1 LEADER]: Becoming Leader. State: Replica: eb428d87d7534482a5d284c35365aeec, State: Running, Role: LEADER
I20260619 02:44:55.091323 29348 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } }
I20260619 02:44:55.091353 29343 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 02:44:55 dist-test-slave-pdnb krb5kdc[29262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:55.092279 29350 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb428d87d7534482a5d284c35365aeec. Latest consensus state: current_term: 1 leader_uuid: "eb428d87d7534482a5d284c35365aeec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } } }
I20260619 02:44:55.092368 29350 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:55.092558 29359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:55.092809 29349 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb428d87d7534482a5d284c35365aeec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb428d87d7534482a5d284c35365aeec" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 35265 } } }
I20260619 02:44:55.093142 29349 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:55.093266 29359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:55.094640 29359 catalog_manager.cc:1347] Generated new cluster ID: f192dff554a949a1ae7957cb1abb0550
I20260619 02:44:55.094689 29359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:55.096257 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:55.085361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33975 (local address 127.7.137.62:35265)
0619 02:44:55.085537 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:55.085546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:55.085581 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:44:55.086068 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:55.086075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:55.086121 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:55.086272 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:55.086888 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:55.088079 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:55.088928 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:55.089339 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:55.089450 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:55.092054 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:55.092057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:55.092058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:55.092069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:55.094581 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:55.095032 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:55.095035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:55.095035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:55.095143 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:55.095389 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:55.095390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:55.095390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:55.095659 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:55.095801 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:55.095966 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:44:55.096057 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260619 02:44:55.097293 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:55.102015 29359 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:55.102456 29359 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:55.114446 29359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb428d87d7534482a5d284c35365aeec: Generated new TSK 0
I20260619 02:44:55.114645 29359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:55.115573 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:55.101083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44311 (local address 127.7.137.62:35265)
0619 02:44:55.101117 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:55.101118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:55.101577 (+ 459us) server_negotiation.cc:407] Connection header received
0619 02:44:55.101792 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:55.101793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:55.101802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:55.101858 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:55.109374 (+ 7516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:55.110163 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:55.111127 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:55.111467 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:55.111533 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:55.112939 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:55.112940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:55.112940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:55.112947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:55.114531 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:55.114922 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:55.114922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:55.114923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:55.114971 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:55.115171 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:55.115171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:55.115171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:55.115261 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:55.115341 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:55.115471 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:44:55.115519 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 02:44:55.117908 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29275
W20260619 02:44:55.122046 29260 connection.cc:535] client connection to 127.7.137.62:35265 recv error: Network error: failed to read from TLS socket (remote: 127.7.137.62:35265): Connection reset by peer (error 104)
2026-06-19T02:44:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:55 dist-test-slave-pdnb krb5kdc[29374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:55 dist-test-slave-pdnb krb5kdc[29374](info): set up 2 sockets
Jun 19 02:44:55 dist-test-slave-pdnb krb5kdc[29374](info): commencing operation
krb5kdc: starting...
W20260619 02:44:56.152815 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:56 dist-test-slave-pdnb krb5kdc[29374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:56.264849 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37955
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:56.344754 29387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:56.344965 29387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:56.344988 29387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:56.346118 29387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:56.346163 29387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:56.346184 29387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:56.346196 29387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:56.346208 29387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:56.347652 29387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:56.347888 29387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:56.348097 29387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:56.352818 29393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:56.352795 29392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:56.352782 29395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:56.352986 29387 server_base.cc:1034] running on GCE node
I20260619 02:44:56.353189 29387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:56.353410 29387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:56.354563 29387 hybrid_clock.cc:648] HybridClock initialized: now 1781837096354554 us; error 27 us; skew 500 ppm
Jun 19 02:44:56 dist-test-slave-pdnb krb5kdc[29374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837096, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:56.360157 29387 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:56.360610 29387 webserver.cc:460] Webserver started at http://127.7.137.62:42423/ using document root <none> and password file <none>
I20260619 02:44:56.360855 29387 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:56.360900 29387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:56.361053 29387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:56.362071 29387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "cf6eac0d6d054386a50a2a043459cb8b"
format_stamp: "Formatted at 2026-06-19 02:44:56 on dist-test-slave-pdnb"
server_key: "1c05bf8dd72118362ad245219ec6811d"
server_key_iv: "c2c5092fc10b3db28706a2694a2087a6"
server_key_version: "encryptionkey@0"
I20260619 02:44:56.362421 29387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "cf6eac0d6d054386a50a2a043459cb8b"
format_stamp: "Formatted at 2026-06-19 02:44:56 on dist-test-slave-pdnb"
server_key: "1c05bf8dd72118362ad245219ec6811d"
server_key_iv: "c2c5092fc10b3db28706a2694a2087a6"
server_key_version: "encryptionkey@0"
I20260619 02:44:56.363660 29387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:56.364323 29402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:56.364478 29387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:56.364557 29387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "cf6eac0d6d054386a50a2a043459cb8b"
format_stamp: "Formatted at 2026-06-19 02:44:56 on dist-test-slave-pdnb"
server_key: "1c05bf8dd72118362ad245219ec6811d"
server_key_iv: "c2c5092fc10b3db28706a2694a2087a6"
server_key_version: "encryptionkey@0"
I20260619 02:44:56.364609 29387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:56.383733 29387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:56.385598 29387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:56.385766 29387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:56.389457 29387 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37955
I20260619 02:44:56.389492 29454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37955 every 8 connection(s)
I20260619 02:44:56.389853 29387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:56.390697 29455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:56.392908 29455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b: Bootstrap starting.
I20260619 02:44:56.393457 29455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:56.393739 29455 log.cc:824] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:56.394385 29455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b: No bootstrap required, opened a new log
I20260619 02:44:56.395918 29455 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } }
I20260619 02:44:56.396051 29455 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:56.396103 29455 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf6eac0d6d054386a50a2a043459cb8b, State: Initialized, Role: FOLLOWER
I20260619 02:44:56.396279 29455 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } }
I20260619 02:44:56.396358 29455 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:56.396404 29455 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:56.396456 29455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:56.397294 29455 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } }
I20260619 02:44:56.397404 29455 leader_election.cc:302] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf6eac0d6d054386a50a2a043459cb8b; no voters:
I20260619 02:44:56.397578 29455 leader_election.cc:288] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:56.397609 29458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:56.397770 29458 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [term 1 LEADER]: Becoming Leader. State: Replica: cf6eac0d6d054386a50a2a043459cb8b, State: Running, Role: LEADER
I20260619 02:44:56.397832 29455 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:56.397898 29458 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } }
I20260619 02:44:56.398237 29459 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf6eac0d6d054386a50a2a043459cb8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } } }
I20260619 02:44:56.398241 29460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf6eac0d6d054386a50a2a043459cb8b. Latest consensus state: current_term: 1 leader_uuid: "cf6eac0d6d054386a50a2a043459cb8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf6eac0d6d054386a50a2a043459cb8b" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37955 } } }
I20260619 02:44:56.398339 29459 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:56.398341 29460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:56.398571 29463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:56.399056 29463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:56.399516 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29387
I20260619 02:44:56.399634 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:56.399814 7716 external_mini_cluster.cc:1428] Setting key 362f95a7fd0b321c00f86f0bb4ecab37
I20260619 02:44:56.400835 29463 catalog_manager.cc:1347] Generated new cluster ID: 33abf363cb0846cda2b32ec842a7861a
I20260619 02:44:56.400884 29463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:56.404639 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:56.400380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59699 (local address 127.7.137.62:37955)
0619 02:44:56.400535 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:56.400544 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:56.400574 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:44:56.400663 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:56.400669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:56.400711 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:56.400882 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:56.401099 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:56.401994 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:56.402755 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:56.403179 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:56.403301 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:56.403975 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:56.403978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:56.403978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:56.404150 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:56.404185 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:56.404196 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:56.404285 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:56.404364 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 02:44:56.404480 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260619 02:44:56.405468 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:56.409541 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:56.406843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35285 (local address 127.7.137.62:37955)
0619 02:44:56.406879 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:56.406880 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:56.406888 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:44:56.406941 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:56.406942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:56.406951 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:56.407004 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:56.407113 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:56.407567 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:56.408262 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:56.408476 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:56.408541 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:56.409293 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:56.409295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:56.409295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:56.409300 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:56.409314 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:56.409318 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:56.409371 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:56.409437 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 02:44:56.409449 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:44:56.424743 29463 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:56.425303 29463 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:56.432214 29463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf6eac0d6d054386a50a2a043459cb8b: Generated new TSK 0
I20260619 02:44:56.432478 29463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:56.436434 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29387
W20260619 02:44:56.440366 29372 connection.cc:535] client connection to 127.7.137.62:37955 recv error: Network error: failed to read from TLS socket (remote: 127.7.137.62:37955): Connection reset by peer (error 104)
2026-06-19T02:44:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:56 dist-test-slave-pdnb krb5kdc[29486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:56 dist-test-slave-pdnb krb5kdc[29486](info): set up 2 sockets
Jun 19 02:44:56 dist-test-slave-pdnb krb5kdc[29486](info): commencing operation
krb5kdc: starting...
W20260619 02:44:57.468930 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:57 dist-test-slave-pdnb krb5kdc[29486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:57.579242 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33989
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:33989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:57.661643 29499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:57.661870 29499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:57.661892 29499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:57.663019 29499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:57.663060 29499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:57.663075 29499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:57.663115 29499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:57.663143 29499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:57.664585 29499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:33989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:57.664847 29499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:57.665068 29499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:57.669627 29505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:57.669718 29507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:57.669718 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:57.669884 29499 server_base.cc:1034] running on GCE node
I20260619 02:44:57.670156 29499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:57.670457 29499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:57.671619 29499 hybrid_clock.cc:648] HybridClock initialized: now 1781837097671612 us; error 37 us; skew 500 ppm
Jun 19 02:44:57 dist-test-slave-pdnb krb5kdc[29486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:57.674039 29499 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:57.674448 29499 webserver.cc:460] Webserver started at http://127.7.137.62:41269/ using document root <none> and password file <none>
I20260619 02:44:57.674638 29499 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:57.674676 29499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:57.674760 29499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:57.675715 29499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "1b3c292195414505828172b965a8a38d"
format_stamp: "Formatted at 2026-06-19 02:44:57 on dist-test-slave-pdnb"
server_key: "17c24504eea4991a654e637c0103a0c4"
server_key_iv: "59c2594cc4e75d039e0c30635cbe6973"
server_key_version: "encryptionkey@0"
I20260619 02:44:57.676012 29499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "1b3c292195414505828172b965a8a38d"
format_stamp: "Formatted at 2026-06-19 02:44:57 on dist-test-slave-pdnb"
server_key: "17c24504eea4991a654e637c0103a0c4"
server_key_iv: "59c2594cc4e75d039e0c30635cbe6973"
server_key_version: "encryptionkey@0"
I20260619 02:44:57.677350 29499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:44:57.678056 29514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:57.678292 29499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:44:57.678382 29499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "1b3c292195414505828172b965a8a38d"
format_stamp: "Formatted at 2026-06-19 02:44:57 on dist-test-slave-pdnb"
server_key: "17c24504eea4991a654e637c0103a0c4"
server_key_iv: "59c2594cc4e75d039e0c30635cbe6973"
server_key_version: "encryptionkey@0"
I20260619 02:44:57.678467 29499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:57.682030 29499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:57.684396 29499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:57.684522 29499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:57.688467 29499 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:33989
I20260619 02:44:57.688534 29566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:33989 every 8 connection(s)
I20260619 02:44:57.688961 29499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:57.689898 29567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:57.692029 29567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: Bootstrap starting.
I20260619 02:44:57.692574 29567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:57.692931 29567 log.cc:824] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:57.693482 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29499
I20260619 02:44:57.693601 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:57.693694 29567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: No bootstrap required, opened a new log
I20260619 02:44:57.693812 7716 external_mini_cluster.cc:1428] Setting key 3de86f2ec48eb3304f6449562b298aee
I20260619 02:44:57.696043 29567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } }
I20260619 02:44:57.696182 29567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:57.696204 29567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3c292195414505828172b965a8a38d, State: Initialized, Role: FOLLOWER
I20260619 02:44:57.696355 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:57.694316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35583 (local address 127.7.137.62:33989)
0619 02:44:57.694472 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:57.694481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:57.694510 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:44:57.694992 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:57.694998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:57.695023 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:57.695130 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:57.695740 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:57.695743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:57.695744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:57.695747 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:57.695784 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:57.695793 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:57.695847 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:57.695923 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:44:57.695980 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260619 02:44:57.696332 29567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } }
I20260619 02:44:57.696448 29567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:57.696481 29567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:57.696529 29567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:57.697273 29567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } }
I20260619 02:44:57.697348 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:57.697377 29567 leader_election.cc:302] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3c292195414505828172b965a8a38d; no voters:
I20260619 02:44:57.697522 29567 leader_election.cc:288] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:57.697568 29572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:57.697708 29572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [term 1 LEADER]: Becoming Leader. State: Replica: 1b3c292195414505828172b965a8a38d, State: Running, Role: LEADER
I20260619 02:44:57.697752 29567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:57.697821 29572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } }
I20260619 02:44:57.698168 29572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3c292195414505828172b965a8a38d. Latest consensus state: current_term: 1 leader_uuid: "1b3c292195414505828172b965a8a38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } } }
I20260619 02:44:57.698381 29572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:57.698304 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3c292195414505828172b965a8a38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3c292195414505828172b965a8a38d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 33989 } } }
I20260619 02:44:57.698760 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d [sys.catalog]: This master's current role is: LEADER
W20260619 02:44:57.699493 29592 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:44:57.699549 29592 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:44:57.699662 29584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:57.699842 29584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:57.700742 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:57.699479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44097 (local address 127.7.137.62:33989)
0619 02:44:57.699510 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:57.699512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:57.699769 (+ 257us) server_negotiation.cc:407] Connection header received
0619 02:44:57.699843 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:57.699843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:57.699848 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:57.699897 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:57.700503 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:57.700504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:57.700504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:44:57.700508 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:57.700521 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:44:57.700524 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:57.700552 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:57.700655 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:44:57.700668 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 02:44:57.701483 29584 catalog_manager.cc:1347] Generated new cluster ID: aae0058255564db7b36f4fc83ecafd83
I20260619 02:44:57.701536 29584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:44:57.749004 29584 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:57.749217 29584 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:57.753583 29584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b3c292195414505828172b965a8a38d: Generated new TSK 0
I20260619 02:44:57.753772 29584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:57.768141 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29499
2026-06-19T02:44:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:57 dist-test-slave-pdnb krb5kdc[29598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:57 dist-test-slave-pdnb krb5kdc[29598](info): set up 2 sockets
Jun 19 02:44:57 dist-test-slave-pdnb krb5kdc[29598](info): commencing operation
krb5kdc: starting...
W20260619 02:44:58.792964 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:44:58 dist-test-slave-pdnb krb5kdc[29598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:44:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:44:58.904417 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37371
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:44:58.984766 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:44:58.984992 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:44:58.985018 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:44:58.986155 29611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:44:58.986189 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:44:58.986204 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:44:58.986217 29611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:44:58.986228 29611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:44:58.987710 29611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:44:58.987901 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:44:58.988066 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:44:58.992823 29616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:58.992960 29611 server_base.cc:1034] running on GCE node
W20260619 02:44:58.992785 29619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:44:58.992798 29617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:44:58.993255 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:44:58.993512 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:44:58.994671 29611 hybrid_clock.cc:648] HybridClock initialized: now 1781837098994670 us; error 38 us; skew 500 ppm
Jun 19 02:44:58 dist-test-slave-pdnb krb5kdc[29598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837098, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:44:58.997040 29611 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:44:58.997519 29611 webserver.cc:460] Webserver started at http://127.7.137.62:38383/ using document root <none> and password file <none>
I20260619 02:44:58.997743 29611 fs_manager.cc:359] Metadata directory not provided
I20260619 02:44:58.997805 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:44:58.997937 29611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:44:58.998984 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "ffced905d7a44a75a983804ddb13db4d"
format_stamp: "Formatted at 2026-06-19 02:44:58 on dist-test-slave-pdnb"
server_key: "d02748bb5bc82534ceca73c0adf1af0d"
server_key_iv: "e531d2c18ff9a2ff01478348303868d9"
server_key_version: "encryptionkey@0"
I20260619 02:44:58.999342 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "ffced905d7a44a75a983804ddb13db4d"
format_stamp: "Formatted at 2026-06-19 02:44:58 on dist-test-slave-pdnb"
server_key: "d02748bb5bc82534ceca73c0adf1af0d"
server_key_iv: "e531d2c18ff9a2ff01478348303868d9"
server_key_version: "encryptionkey@0"
I20260619 02:44:59.000655 29611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:44:59.001370 29626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:44:59.001535 29611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:44:59.001643 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "ffced905d7a44a75a983804ddb13db4d"
format_stamp: "Formatted at 2026-06-19 02:44:58 on dist-test-slave-pdnb"
server_key: "d02748bb5bc82534ceca73c0adf1af0d"
server_key_iv: "e531d2c18ff9a2ff01478348303868d9"
server_key_version: "encryptionkey@0"
I20260619 02:44:59.001722 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:44:59.013975 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:44:59.015679 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:44:59.015827 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:44:59.019723 29611 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37371
I20260619 02:44:59.019768 29678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37371 every 8 connection(s)
I20260619 02:44:59.020105 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:44:59.020933 29679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:44:59.023046 29679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d: Bootstrap starting.
I20260619 02:44:59.023594 29679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d: Neither blocks nor log segments found. Creating new log.
I20260619 02:44:59.023855 29679 log.cc:824] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:44:59.024477 29679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d: No bootstrap required, opened a new log
I20260619 02:44:59.026060 29679 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:44:59.026197 29679 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:44:59.026221 29679 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffced905d7a44a75a983804ddb13db4d, State: Initialized, Role: FOLLOWER
I20260619 02:44:59.026391 29679 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:44:59.026470 29679 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:44:59.026517 29679 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:44:59.026567 29679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:44:59.027336 29679 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:44:59.027442 29679 leader_election.cc:302] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffced905d7a44a75a983804ddb13db4d; no voters:
I20260619 02:44:59.027603 29679 leader_election.cc:288] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:44:59.027652 29682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:44:59.027810 29682 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [term 1 LEADER]: Becoming Leader. State: Replica: ffced905d7a44a75a983804ddb13db4d, State: Running, Role: LEADER
I20260619 02:44:59.027840 29679 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:44:59.027908 29682 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } }
I20260619 02:44:59.028301 29684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffced905d7a44a75a983804ddb13db4d. Latest consensus state: current_term: 1 leader_uuid: "ffced905d7a44a75a983804ddb13db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } } }
I20260619 02:44:59.028410 29684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:59.028285 29683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffced905d7a44a75a983804ddb13db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffced905d7a44a75a983804ddb13db4d" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37371 } } }
I20260619 02:44:59.028616 29683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d [sys.catalog]: This master's current role is: LEADER
I20260619 02:44:59.028750 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29611
I20260619 02:44:59.028867 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:44:59.029021 29691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:44:59.029075 7716 external_mini_cluster.cc:1428] Setting key fa0d629171e20f1ee4e059ea87db8527
I20260619 02:44:59.029696 29691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:44:59.031134 29691 catalog_manager.cc:1347] Generated new cluster ID: 87910b0d4c61461e94ee1a338593b320
I20260619 02:44:59.031183 29691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 02:44:59 dist-test-slave-pdnb krb5kdc[29598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:44:59.040227 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:59.029505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43319 (local address 127.7.137.62:37371)
0619 02:44:59.029795 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:59.029805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:59.029837 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:44:59.029946 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:59.029953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:59.030007 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:59.030166 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:59.030899 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:59.032234 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:59.033302 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:59.033906 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:59.033994 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:59.036708 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:59.036712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:59.036712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:59.036745 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:59.038671 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:59.039032 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:59.039035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:59.039035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:59.039150 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:59.039339 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:59.039340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:59.039340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:59.039646 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:59.039795 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:59.039889 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 02:44:59.040020 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":66,"threads_started":1}
I20260619 02:44:59.041142 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:44:59.049319 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:44:59.042532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47009 (local address 127.7.137.62:37371)
0619 02:44:59.042569 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:44:59.042571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:44:59.042735 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:44:59.042777 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:44:59.042778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:44:59.042789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:44:59.042850 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:44:59.043406 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:59.044176 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:59.045123 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:44:59.045434 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:44:59.045498 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:44:59.046719 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:44:59.046720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:44:59.046720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:44:59.046727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:44:59.048231 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:59.048579 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:59.048580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:59.048580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:59.048642 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:44:59.048845 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:44:59.048845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:44:59.048846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:44:59.048968 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:44:59.049061 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:44:59.049159 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:44:59.049229 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:44:59.051393 29691 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:44:59.051864 29691 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:44:59.053831 29691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffced905d7a44a75a983804ddb13db4d: Generated new TSK 0
I20260619 02:44:59.053996 29691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:44:59.055593 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29611
2026-06-19T02:44:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:44:59 dist-test-slave-pdnb krb5kdc[29710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:44:59 dist-test-slave-pdnb krb5kdc[29710](info): set up 2 sockets
Jun 19 02:44:59 dist-test-slave-pdnb krb5kdc[29710](info): commencing operation
krb5kdc: starting...
W20260619 02:45:00.085011 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:45:00 dist-test-slave-pdnb krb5kdc[29710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:45:00.201275 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:43933
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:43933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:00.282151 29723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:00.282399 29723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:00.282437 29723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:00.283576 29723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:00.283644 29723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:00.283669 29723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:00.283697 29723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:00.283723 29723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:00.285200 29723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:43933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:43933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:00.285475 29723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:00.285688 29723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:00.290704 29728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:00.290763 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:00.290719 29723 server_base.cc:1034] running on GCE node
W20260619 02:45:00.290778 29729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:00.291113 29723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:00.291417 29723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:00.292588 29723 hybrid_clock.cc:648] HybridClock initialized: now 1781837100292568 us; error 36 us; skew 500 ppm
Jun 19 02:45:00 dist-test-slave-pdnb krb5kdc[29710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837100, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:00.295171 29723 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:45:00.295684 29723 webserver.cc:460] Webserver started at http://127.7.137.62:42119/ using document root <none> and password file <none>
I20260619 02:45:00.295905 29723 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:00.295955 29723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:00.296096 29723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:00.297252 29723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe"
format_stamp: "Formatted at 2026-06-19 02:45:00 on dist-test-slave-pdnb"
server_key: "2637319ab4b1b827345efa32669ee5b5"
server_key_iv: "b98de91c281b535db5a43ce732d99456"
server_key_version: "encryptionkey@0"
I20260619 02:45:00.297616 29723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe"
format_stamp: "Formatted at 2026-06-19 02:45:00 on dist-test-slave-pdnb"
server_key: "2637319ab4b1b827345efa32669ee5b5"
server_key_iv: "b98de91c281b535db5a43ce732d99456"
server_key_version: "encryptionkey@0"
I20260619 02:45:00.298911 29723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:00.299638 29738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:00.299798 29723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:00.299901 29723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe"
format_stamp: "Formatted at 2026-06-19 02:45:00 on dist-test-slave-pdnb"
server_key: "2637319ab4b1b827345efa32669ee5b5"
server_key_iv: "b98de91c281b535db5a43ce732d99456"
server_key_version: "encryptionkey@0"
I20260619 02:45:00.299978 29723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:00.310140 29723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:00.311892 29723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:00.312047 29723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:00.315837 29723 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:43933
I20260619 02:45:00.315866 29790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:43933 every 8 connection(s)
I20260619 02:45:00.316197 29723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:00.317099 29791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:00.319270 29791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe: Bootstrap starting.
I20260619 02:45:00.319832 29791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:00.320111 29791 log.cc:824] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:00.320788 29791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe: No bootstrap required, opened a new log
I20260619 02:45:00.322326 29791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } }
I20260619 02:45:00.322459 29791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:00.322500 29791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc4d102bdcf4629bffd4bf0f60ffbbe, State: Initialized, Role: FOLLOWER
I20260619 02:45:00.322643 29791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } }
I20260619 02:45:00.322705 29791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:00.322726 29791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:00.322793 29791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:00.323587 29791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } }
I20260619 02:45:00.323693 29791 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc4d102bdcf4629bffd4bf0f60ffbbe; no voters:
I20260619 02:45:00.323885 29791 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:00.323908 29794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:00.324021 29794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [term 1 LEADER]: Becoming Leader. State: Replica: 4cc4d102bdcf4629bffd4bf0f60ffbbe, State: Running, Role: LEADER
I20260619 02:45:00.324155 29791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:00.324159 29794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } }
I20260619 02:45:00.324610 29795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } } }
I20260619 02:45:00.324630 29796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc4d102bdcf4629bffd4bf0f60ffbbe. Latest consensus state: current_term: 1 leader_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4d102bdcf4629bffd4bf0f60ffbbe" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 43933 } } }
I20260619 02:45:00.324854 29796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:00.324764 29795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:00.325310 29803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:00.325663 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29723
I20260619 02:45:00.325779 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:45:00.325845 29803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:00.325982 7716 external_mini_cluster.cc:1428] Setting key 0c1d1bb09e9b920d1e74d0184cb4cf9f
I20260619 02:45:00.329180 29803 catalog_manager.cc:1347] Generated new cluster ID: 31f7fff5c9004af9ac09d65bd9651f97
I20260619 02:45:00.329237 29803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:00.334637 29803 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:00.335021 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:00.326403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42831 (local address 127.7.137.62:43933)
0619 02:45:00.326585 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:00.326594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:00.326728 (+ 134us) server_negotiation.cc:407] Connection header received
0619 02:45:00.326869 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:00.326876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:00.326962 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:00.327119 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:00.327308 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:00.328158 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:00.332994 (+ 4836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:00.333444 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:00.333566 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:00.334264 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:00.334268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:00.334268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:00.334455 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:00.334507 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:00.334522 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:00.334593 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:00.334732 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:45:00.334802 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260619 02:45:00.335564 29803 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:00.335840 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:00.339975 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:00.337305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51845 (local address 127.7.137.62:43933)
0619 02:45:00.337341 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:00.337343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:00.337367 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:45:00.337386 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:00.337386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:00.337396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:00.337456 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:00.337593 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:00.338065 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:00.338785 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:00.339002 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:00.339074 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:00.339735 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:00.339737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:00.339737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:00.339741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:00.339755 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:00.339759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:00.339812 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:00.339862 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 02:45:00.339887 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:00.347797 29803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cc4d102bdcf4629bffd4bf0f60ffbbe: Generated new TSK 0
I20260619 02:45:00.348001 29803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:00.350057 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29723
2026-06-19T02:45:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:00 dist-test-slave-pdnb krb5kdc[29822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:00 dist-test-slave-pdnb krb5kdc[29822](info): set up 2 sockets
Jun 19 02:45:00 dist-test-slave-pdnb krb5kdc[29822](info): commencing operation
krb5kdc: starting...
W20260619 02:45:01.385035 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:45:01 dist-test-slave-pdnb krb5kdc[29822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:45:01.497430 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36059
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:35071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:01.576099 29835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:01.576335 29835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:01.576372 29835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:01.577534 29835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:01.577607 29835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:01.577636 29835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:01.577666 29835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:01.577690 29835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:01.579206 29835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:35071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:01.579463 29835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:01.579676 29835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:01.584342 29841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:01.584375 29843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:01.584353 29840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:01.584419 29835 server_base.cc:1034] running on GCE node
I20260619 02:45:01.584826 29835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:01.585142 29835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:01.586300 29835 hybrid_clock.cc:648] HybridClock initialized: now 1781837101586292 us; error 38 us; skew 500 ppm
Jun 19 02:45:01 dist-test-slave-pdnb krb5kdc[29822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837101, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:01.592631 29835 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:45:01.593072 29835 webserver.cc:460] Webserver started at http://127.7.137.62:39123/ using document root <none> and password file <none>
I20260619 02:45:01.593307 29835 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:01.593369 29835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:01.593509 29835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:01.594551 29835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "df7c6da3b39d476f979ee9a28ad6fc6e"
format_stamp: "Formatted at 2026-06-19 02:45:01 on dist-test-slave-pdnb"
server_key: "5ff1e696442eb71695ff94dfd92986fa"
server_key_iv: "ed3e835187e83bd18b7c6d934417d965"
server_key_version: "encryptionkey@0"
I20260619 02:45:01.594914 29835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "df7c6da3b39d476f979ee9a28ad6fc6e"
format_stamp: "Formatted at 2026-06-19 02:45:01 on dist-test-slave-pdnb"
server_key: "5ff1e696442eb71695ff94dfd92986fa"
server_key_iv: "ed3e835187e83bd18b7c6d934417d965"
server_key_version: "encryptionkey@0"
I20260619 02:45:01.596226 29835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 02:45:01.597127 29850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:01.597290 29835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 02:45:01.597407 29835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "df7c6da3b39d476f979ee9a28ad6fc6e"
format_stamp: "Formatted at 2026-06-19 02:45:01 on dist-test-slave-pdnb"
server_key: "5ff1e696442eb71695ff94dfd92986fa"
server_key_iv: "ed3e835187e83bd18b7c6d934417d965"
server_key_version: "encryptionkey@0"
I20260619 02:45:01.597487 29835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:01.600903 29835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:01.603132 29835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:01.603281 29835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:01.607223 29835 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36059
I20260619 02:45:01.607251 29902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36059 every 8 connection(s)
I20260619 02:45:01.607581 29835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:01.608496 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:01.610709 29903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e: Bootstrap starting.
I20260619 02:45:01.611269 29903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:01.611538 29903 log.cc:824] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:01.611868 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29835
I20260619 02:45:01.611985 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:45:01.612164 7716 external_mini_cluster.cc:1428] Setting key 75dbccbc6e049d3cbfd5bef5f303acd0
I20260619 02:45:01.612265 29903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e: No bootstrap required, opened a new log
I20260619 02:45:01.614552 29903 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } }
I20260619 02:45:01.614739 29903 raft_consensus.cc:374] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:45:01.614811 29903 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7c6da3b39d476f979ee9a28ad6fc6e, State: Initialized, Role: FOLLOWER
I20260619 02:45:01.614945 29903 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } }
I20260619 02:45:01.615044 29903 raft_consensus.cc:388] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:45:01.615051 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:01.612670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57099 (local address 127.7.137.62:36059)
0619 02:45:01.612856 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:01.612865 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:01.612899 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:45:01.613668 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:01.613677 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:01.613713 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:01.613855 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:01.614474 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:01.614478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:01.614478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:01.614482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:01.614528 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:01.614543 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:01.614620 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:01.614657 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:45:01.614695 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260619 02:45:01.615100 29903 raft_consensus.cc:482] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:45:01.615146 29903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:01.615711 29903 raft_consensus.cc:504] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } }
I20260619 02:45:01.615831 29903 leader_election.cc:302] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df7c6da3b39d476f979ee9a28ad6fc6e; no voters:
I20260619 02:45:01.616021 29903 leader_election.cc:288] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:45:01.616040 29908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:01.616075 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:01.616196 29908 raft_consensus.cc:686] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [term 1 LEADER]: Becoming Leader. State: Replica: df7c6da3b39d476f979ee9a28ad6fc6e, State: Running, Role: LEADER
I20260619 02:45:01.616322 29903 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:01.616323 29908 consensus_queue.cc:237] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } }
I20260619 02:45:01.616806 29909 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader df7c6da3b39d476f979ee9a28ad6fc6e. Latest consensus state: current_term: 1 leader_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } } }
I20260619 02:45:01.616923 29909 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:01.617153 29910 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c6da3b39d476f979ee9a28ad6fc6e" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36059 } } }
I20260619 02:45:01.617525 29910 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:01.617712 29915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:01.618620 29915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:01.618745 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:01.617572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59727 (local address 127.7.137.62:36059)
0619 02:45:01.617606 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:01.617608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:01.617739 (+ 131us) server_negotiation.cc:407] Connection header received
0619 02:45:01.617834 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:01.617836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:01.617840 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:01.617869 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:01.618462 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:01.618463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:01.618464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:45:01.618468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:01.618482 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:45:01.618487 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:01.618541 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:01.618658 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:45:01.618668 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:01.620025 29915 catalog_manager.cc:1347] Generated new cluster ID: 551378201bdc4277b17e9b7293d1cf6e
I20260619 02:45:01.620096 29915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:01.634716 29915 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:01.634894 29915 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:01.644495 29915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df7c6da3b39d476f979ee9a28ad6fc6e: Generated new TSK 0
I20260619 02:45:01.644691 29915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:01.656855 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29835
2026-06-19T02:45:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1307 ms)
[----------] 18 tests from AuthTokenIssuingTest (23692 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:01 dist-test-slave-pdnb krb5kdc[29934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:01 dist-test-slave-pdnb krb5kdc[29934](info): set up 2 sockets
Jun 19 02:45:01 dist-test-slave-pdnb krb5kdc[29934](info): commencing operation
krb5kdc: starting...
W20260619 02:45:02.688999 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:45:02 dist-test-slave-pdnb krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:45:02.803367 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37165
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:02.883123 29947 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:02.883327 29947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:02.883345 29947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:02.883390 29947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:02.884521 29947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:02.884559 29947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:02.884574 29947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:02.884621 29947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:02.884645 29947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:02.886155 29947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:37165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:02.886394 29947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:02.886617 29947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:02.891341 29952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:02.891388 29955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:02.891517 29947 server_base.cc:1034] running on GCE node
W20260619 02:45:02.891386 29953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:02.891848 29947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:02.892134 29947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:02.893285 29947 hybrid_clock.cc:648] HybridClock initialized: now 1781837102893265 us; error 33 us; skew 500 ppm
Jun 19 02:45:02 dist-test-slave-pdnb krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:02.895589 29947 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:45:02.895990 29947 webserver.cc:460] Webserver started at http://127.7.137.62:34337/ using document root <none> and password file <none>
I20260619 02:45:02.896229 29947 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:02.896293 29947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:02.896432 29947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:02.897547 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "e69d72ae85ca4e40b41a44fd5d29b319"
format_stamp: "Formatted at 2026-06-19 02:45:02 on dist-test-slave-pdnb"
server_key: "c43221dae8ae04a769401268a78ede3c"
server_key_iv: "666c26534a6415cde86386604a40a2e1"
server_key_version: "encryptionkey@0"
I20260619 02:45:02.897893 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "e69d72ae85ca4e40b41a44fd5d29b319"
format_stamp: "Formatted at 2026-06-19 02:45:02 on dist-test-slave-pdnb"
server_key: "c43221dae8ae04a769401268a78ede3c"
server_key_iv: "666c26534a6415cde86386604a40a2e1"
server_key_version: "encryptionkey@0"
I20260619 02:45:02.899160 29947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:02.899829 29962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:02.899986 29947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:02.900087 29947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "e69d72ae85ca4e40b41a44fd5d29b319"
format_stamp: "Formatted at 2026-06-19 02:45:02 on dist-test-slave-pdnb"
server_key: "c43221dae8ae04a769401268a78ede3c"
server_key_iv: "666c26534a6415cde86386604a40a2e1"
server_key_version: "encryptionkey@0"
I20260619 02:45:02.900156 29947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:02.941841 29947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:02.944612 29947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:02.944783 29947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:02.949610 29947 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:37165
I20260619 02:45:02.949682 30014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:37165 every 8 connection(s)
I20260619 02:45:02.950028 29947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:02.951272 30015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:02.953927 30015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:02.958281 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 29947
I20260619 02:45:02.958392 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:45:02.958575 7716 external_mini_cluster.cc:1428] Setting key ee180bf0c2842e8d436a38428da4f416
Jun 19 02:45:02 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:02.966217 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:02.954794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37165 (local address 127.0.0.1:44534)
0619 02:45:02.954997 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:02.955049 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:02.955761 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:02.956229 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:02.956232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:02.956578 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:02.956828 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:02.956833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:02.958505 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:02.958505 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:02.959722 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:02.959725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:02.960572 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:02.960573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:02.960653 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:02.961252 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:02.961263 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:02.962793 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:02.964830 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:02.964837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:02.964838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:02.965028 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:02.965287 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:02.965288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:02.965288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:02.965361 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:02.965786 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:02.965790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:02.965842 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:45:02.965913 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:45:02.966040 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.7.137.61@KRBTEST.COM; defaulting to no policy
I20260619 02:45:02.966240 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:02.954825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.7.137.62:37165)
0619 02:45:02.955954 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:02.955959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:02.955977 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:45:02.956066 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:02.956068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:02.956091 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:02.956179 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:02.956931 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:02.958383 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:02.959817 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:02.960336 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:02.960452 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:02.962875 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:02.962883 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:02.962883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:02.962891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:02.964754 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:02.965108 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:02.965112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:02.965112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:02.965216 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:02.965422 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:02.965423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:02.965423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:02.965712 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:02.965809 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:02.965969 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:45:02.966116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":1102,"thread_start_us":74,"threads_started":1}
I20260619 02:45:02.966917 30015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:02.967257 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:02.967132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59644)
0619 02:45:02.967197 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:02.967242 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 02:45:02.967379 29963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
I20260619 02:45:02.967742 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:02.967669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59656)
0619 02:45:02.967707 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:02.967728 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:45:02.967835 30015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:42569: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
I20260619 02:45:02.981487 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:42569
--webserver_interface=127.7.137.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:03.017052 30015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } attempt: 1
I20260619 02:45:03.017614 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.017443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59664)
0619 02:45:03.017515 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.017568 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:45:03.018011 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.017897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59674)
0619 02:45:03.017947 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.017980 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:45:03.018148 30015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:42569: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
W20260619 02:45:03.062357 30022 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:03.062564 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:03.062582 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:03.062601 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:03.063709 30022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:03.063748 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:03.063762 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:03.063776 30022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:03.063822 30022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:03.065313 30022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:42569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.137.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:03.065544 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:03.065757 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:03.070366 30030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:03.070374 30027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:03.070403 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:03.070676 30022 server_base.cc:1034] running on GCE node
I20260619 02:45:03.070911 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:03.071178 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:03.072314 30022 hybrid_clock.cc:648] HybridClock initialized: now 1781837103072288 us; error 44 us; skew 500 ppm
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:03.074849 30022 init.cc:377] Logged in from keytab as kudu/127.7.137.61@KRBTEST.COM (short username kudu)
I20260619 02:45:03.075300 30022 webserver.cc:460] Webserver started at http://127.7.137.61:46081/ using document root <none> and password file <none>
I20260619 02:45:03.075541 30022 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:03.075584 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:03.075670 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:03.076654 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data/instance:
uuid: "672d727ac43146aebe34daefacb4d9c7"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "a056cd72c8d42db633db0edf2cee48b3"
server_key_iv: "882d97e431ab07675d3d419a14bf79b1"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.077024 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal/instance:
uuid: "672d727ac43146aebe34daefacb4d9c7"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "a056cd72c8d42db633db0edf2cee48b3"
server_key_iv: "882d97e431ab07675d3d419a14bf79b1"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.078227 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:03.078868 30037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:03.079030 30022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:03.079125 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "672d727ac43146aebe34daefacb4d9c7"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "a056cd72c8d42db633db0edf2cee48b3"
server_key_iv: "882d97e431ab07675d3d419a14bf79b1"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.079196 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:03.086308 30015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } attempt: 2
I20260619 02:45:03.086709 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.086576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59682)
0619 02:45:03.086631 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.086672 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:45:03.087055 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.086960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59688)
0619 02:45:03.087000 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.087027 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:45:03.087193 30015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:42569: Network error: Client connection negotiation failed: client connection to 127.7.137.61:42569: connect: Connection refused (error 111)
I20260619 02:45:03.093787 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:03.095507 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:03.095667 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:03.099563 30022 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.61:42569
I20260619 02:45:03.099593 30089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.61:42569 every 8 connection(s)
I20260619 02:45:03.099942 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
I20260619 02:45:03.100930 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:03.102506 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:03.105844 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 30022
I20260619 02:45:03.105962 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-1/wal/instance
I20260619 02:45:03.106134 7716 external_mini_cluster.cc:1428] Setting key 8a7ce758e2fe079c19f124f506c46299
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:03.111430 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.103170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.7.137.62:37165)
0619 02:45:03.103244 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.103247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.103444 (+ 197us) server_negotiation.cc:407] Connection header received
0619 02:45:03.103869 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.103870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.103882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.103931 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.104500 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.105044 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.106007 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.106264 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.106335 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.108411 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.108412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.108413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.108420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.109928 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.110437 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.110438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.110439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.110494 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.110759 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.110760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.110760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.110872 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.110938 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.111256 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.111340 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:45:03.111606 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.103144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37165 (local address 127.0.0.1:44542)
0619 02:45:03.103325 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.103369 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.103793 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.104013 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.104016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.104249 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.104432 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.104436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.105116 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.105117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.105892 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.105894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.106365 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.106367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.106486 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.106981 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.106989 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.108312 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.110013 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.110020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.110022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.110360 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.110574 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.110575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.110575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.110686 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.110941 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.110966 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.111122 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.111222 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.111375 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260619 02:45:03.112146 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.137.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.60@KRBTEST.COM" created.
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:45:03.119530 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.112472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.7.137.61:42569)
0619 02:45:03.112629 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.112633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.112647 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:45:03.112663 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.112664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.112674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.112758 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.113015 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.113720 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.114488 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.114800 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.114884 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.116686 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.116693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.116694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.116701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.118314 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.118696 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.118699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.118699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.118748 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.118938 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.118938 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.118938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.119061 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.119122 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.119304 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.119419 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260619 02:45:03.119550 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.112403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59694)
0619 02:45:03.112471 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.112480 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.112518 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.112777 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.112778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.112906 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.112947 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.112948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.113800 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.113801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.114400 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.114402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.114883 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.114883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.114945 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.115344 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.115348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.116608 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.118394 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.118395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.118395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.118622 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.118807 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.118808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.118808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.118879 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.119126 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.119127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.119178 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.119260 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.119371 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 02:45:03.120237 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:03.120571 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.120446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41766)
0619 02:45:03.120491 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.120539 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:45:03.120769 30038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
I20260619 02:45:03.121202 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.121096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41768)
0619 02:45:03.121140 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.121170 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 02:45:03.121311 30090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:38579: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
I20260619 02:45:03.129405 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:38579
--webserver_interface=127.7.137.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:03.170564 30090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } attempt: 1
I20260619 02:45:03.171113 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.170968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41774)
0619 02:45:03.171036 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.171082 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:45:03.171505 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.171405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41782)
0619 02:45:03.171456 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.171481 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:45:03.171612 30090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:38579: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
I20260619 02:45:03.178344 30015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } attempt: 3
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:45:03.185410 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.178729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.7.137.61:42569)
0619 02:45:03.178784 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.178787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.178818 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:45:03.178847 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.178848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.178856 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.178918 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.179237 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.179732 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.180429 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.180644 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.180718 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.182833 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.182835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.182835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.182842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.184325 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.184701 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.184702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.184702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.184800 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.184989 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.184989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.184989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.185092 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.185139 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.185287 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.185347 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:45:03.185465 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.178663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59702)
0619 02:45:03.178717 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.178726 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.178780 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.178924 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.178926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.179092 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.179160 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.179162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.179816 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.179818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.180348 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.180350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.180715 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.180716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.180810 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.181251 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.181257 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.182728 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.184401 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.184402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.184403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.184631 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.184858 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.184859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.184860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.184926 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.185145 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.185146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.185197 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.185271 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.185365 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 02:45:03.185891 30015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:03.186224 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.186113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41786)
0619 02:45:03.186159 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.186194 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:45:03.186535 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.186440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41798)
0619 02:45:03.186486 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.186512 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 02:45:03.186671 30015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:38579: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
W20260619 02:45:03.209750 30097 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:03.209944 30097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:03.209960 30097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:03.209980 30097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:03.211002 30097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:03.211040 30097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:03.211055 30097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:03.211067 30097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:03.211081 30097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:03.212442 30097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:37165,127.7.137.61:42569,127.7.137.60:38579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:38579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.137.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:03.212615 30097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:03.212834 30097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:03.217288 30103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:03.217321 30102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:03.217288 30105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:03.217603 30097 server_base.cc:1034] running on GCE node
I20260619 02:45:03.217787 30015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } attempt: 1
I20260619 02:45:03.217887 30097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:03.218133 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.218030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41810)
0619 02:45:03.218072 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.218104 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 02:45:03.218214 30097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:03.218525 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.218423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41822)
0619 02:45:03.218471 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.218500 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 02:45:03.218621 30015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:38579: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
I20260619 02:45:03.219368 30097 hybrid_clock.cc:648] HybridClock initialized: now 1781837103219357 us; error 26 us; skew 500 ppm
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:03.221699 30097 init.cc:377] Logged in from keytab as kudu/127.7.137.60@KRBTEST.COM (short username kudu)
I20260619 02:45:03.222116 30097 webserver.cc:460] Webserver started at http://127.7.137.60:41683/ using document root <none> and password file <none>
I20260619 02:45:03.222342 30097 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:03.222385 30097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:03.222543 30097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:03.223593 30097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data/instance:
uuid: "f5d2d156ac8748718be69c6877edc88f"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "ce5e5c78a349b8dd2cbc57191c72b711"
server_key_iv: "86700484c1318c03bbd461017a06c945"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.223930 30097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal/instance:
uuid: "f5d2d156ac8748718be69c6877edc88f"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "ce5e5c78a349b8dd2cbc57191c72b711"
server_key_iv: "86700484c1318c03bbd461017a06c945"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.225448 30097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:03.226154 30112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:03.226334 30097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:03.226415 30097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "f5d2d156ac8748718be69c6877edc88f"
format_stamp: "Formatted at 2026-06-19 02:45:03 on dist-test-slave-pdnb"
server_key: "ce5e5c78a349b8dd2cbc57191c72b711"
server_key_iv: "86700484c1318c03bbd461017a06c945"
server_key_version: "encryptionkey@0"
I20260619 02:45:03.226485 30097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:03.239748 30090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } attempt: 2
I20260619 02:45:03.240161 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.240042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41838)
0619 02:45:03.240095 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.240132 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 02:45:03.240556 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.240475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41842)
0619 02:45:03.240511 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.240540 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 02:45:03.240665 30090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:38579: Network error: Client connection negotiation failed: client connection to 127.7.137.60:38579: connect: Connection refused (error 111)
I20260619 02:45:03.260080 30097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:03.262063 30097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:03.262202 30097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:03.266147 30097 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.60:38579
I20260619 02:45:03.266209 30164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.60:38579 every 8 connection(s)
I20260619 02:45:03.266543 30097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
I20260619 02:45:03.267506 30165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:03.269115 30165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:03.273885 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 30097
I20260619 02:45:03.274003 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836839601721-7716-0/minicluster-data/master-2/wal/instance
I20260619 02:45:03.274179 7716 external_mini_cluster.cc:1428] Setting key e4747652896392f706967d3336589d3b
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:03.278832 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.269807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.7.137.62:37165)
0619 02:45:03.269934 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.269937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.270100 (+ 163us) server_negotiation.cc:407] Connection header received
0619 02:45:03.270594 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.270595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.270607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.270666 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.271277 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.271775 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.272860 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.273055 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.273116 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.275533 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.275535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.275535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.275560 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.277417 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.277852 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.277853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.277853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.277916 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.278220 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.278220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.278221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.278325 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.278411 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.278697 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.278767 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:45:03.279115 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.269766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:37165 (local address 127.0.0.1:44548)
0619 02:45:03.269971 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.270015 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.270489 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.270739 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.270742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.270991 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.271181 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.271185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.271909 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.271910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.272698 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.272700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.273147 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.273148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.273257 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.273713 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.273722 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.275447 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.277497 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.277504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.277506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.277788 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.278009 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.278010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.278010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.278154 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.278408 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.278413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.278551 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.278673 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.278833 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260619 02:45:03.279700 30165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:03.283811 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.274550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.7.137.60:38579)
0619 02:45:03.274851 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.274855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.274869 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:45:03.274886 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.274887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.274903 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.274974 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.275980 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.276851 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.277649 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.278029 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.278088 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.280633 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.280641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.280641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.280648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.282471 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.282910 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.282913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.282913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.282985 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.283171 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.283172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.283172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.283328 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.283536 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.283584 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.283685 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":78,"threads_started":1}
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:45:03.286799 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.279960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:42569 (local address 127.0.0.1:59708)
0619 02:45:03.280004 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.280014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.280054 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.280174 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.280174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.280322 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.280365 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.280366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.281276 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.281277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.281979 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.281981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.282526 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.282527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.282599 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.282992 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.282997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.284353 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.286014 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.286015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.286015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.286170 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.286353 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.286354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.286354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.286397 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.286616 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.286617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.286644 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.286687 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.286745 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 02:45:03.286895 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.280014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.7.137.61:42569)
0619 02:45:03.280067 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.280069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.280078 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:03.280106 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.280107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.280116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.280171 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.280466 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.281189 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.282068 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.282432 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.282515 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.284467 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.284468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.284468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.284474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.285938 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.286231 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.286232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.286232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.286291 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.286446 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.286447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.286447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.286563 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.286628 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.286722 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.286802 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 02:45:03.287175 30165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:03.292107 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.284580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.7.137.62:37165)
0619 02:45:03.284616 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.284618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.284625 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:03.284683 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.284683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.284690 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.284785 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.285426 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.285888 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.286788 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.287086 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.287146 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.289440 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.289441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.289441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.289446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.290958 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.291297 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.291298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.291299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.291342 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.291555 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.291556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.291557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.291688 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.291859 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.291897 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.291988 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:03.293720 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.287410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41874)
0619 02:45:03.287491 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.287500 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.287546 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.287651 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.287652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.287794 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.287829 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.287830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.288442 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.288444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.289017 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.289019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.289370 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.289371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.289447 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.289775 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.289778 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.290960 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.292762 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.292762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.292763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.292966 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.293156 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.293157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.293157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.293208 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.293428 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.293429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.293463 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.293532 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.293620 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 02:45:03.293730 30015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } attempt: 2
I20260619 02:45:03.293874 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.287452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.7.137.60:38579)
0619 02:45:03.287485 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.287487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.287532 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:45:03.287598 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.287599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.287606 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.287644 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.287899 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.288368 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.289116 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.289303 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.289347 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.291059 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.291060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.291060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.291066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.292630 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.293045 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.293046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.293046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.293093 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.293270 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.293271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.293271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.293370 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.293437 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.293551 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.293666 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":18}
I20260619 02:45:03.295202 30165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: Bootstrap starting.
I20260619 02:45:03.296181 30165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:03.296484 30165 log.cc:824] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:03.298475 30165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: No bootstrap required, opened a new log
I20260619 02:45:03.300837 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.292862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.7.137.61:42569)
0619 02:45:03.292900 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.292902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.292908 (+ 6us) server_negotiation.cc:407] Connection header received
0619 02:45:03.292927 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.292928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.292939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.292995 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.293440 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.294276 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.295142 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.295437 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.295467 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.298048 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.298049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.298050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.298057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.299638 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.299980 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.299981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.299982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.300039 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.300239 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.300240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.300241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.300391 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.300561 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.300633 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.300710 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:03.300823 30165 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.301029 30165 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d2d156ac8748718be69c6877edc88f, State: Initialized, Role: FOLLOWER
I20260619 02:45:03.301122 30165 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.301222 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.294027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41880)
0619 02:45:03.294098 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.294107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.294142 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.294328 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.294329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.294476 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.294516 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.294518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.295357 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.295358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.296146 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.296149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.296674 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.296675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.296758 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.297133 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.297139 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.298336 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.300057 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.300058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.300058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.300273 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.300496 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.300496 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.300497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.300556 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.300914 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.300915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.300981 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.301040 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.301130 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:45:03.301450 30165 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:03.301566 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.294182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.7.137.60:38579)
0619 02:45:03.294229 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.294231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.294239 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:03.294257 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.294257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.294265 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.294326 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.294594 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.295258 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.296235 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.296575 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.296673 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.298441 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.298442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.298443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.298448 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.299965 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.300344 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.300345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.300345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.300396 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.300625 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.300626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.300626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.300823 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.300884 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.301403 (+ 519us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.301478 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:45:03.301566 30171 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.301664 30171 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.302256 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:03.303346 30015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: Bootstrap starting.
W20260619 02:45:03.303522 30182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:03.303618 30182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:03.303278 29989 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e69d72ae85ca4e40b41a44fd5d29b319"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44564
I20260619 02:45:03.304255 30015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:03.304561 30015 log.cc:824] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:03.305408 30015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: No bootstrap required, opened a new log
I20260619 02:45:03.306785 30015 raft_consensus.cc:348] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.306916 30015 raft_consensus.cc:729] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69d72ae85ca4e40b41a44fd5d29b319, State: Initialized, Role: FOLLOWER
I20260619 02:45:03.307008 30015 consensus_queue.cc:260] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.307301 30015 sys_catalog.cc:564] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:03.307276 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.307355 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.305205 30097 master_runner.cc:186] Error getting master registration for 127.7.137.62:37165: OK, instance_id { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" instance_seqno: 1781837102948330 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:45:03.308305 30097 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:45:03.308465 29989 raft_consensus.cc:482] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:45:03.308532 29989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:03.309396 29989 raft_consensus.cc:504] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.309692 29989 leader_election.cc:288] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [CANDIDATE]: Term 1 election: Requested vote from peers 672d727ac43146aebe34daefacb4d9c7 (127.7.137.61:42569), f5d2d156ac8748718be69c6877edc88f (127.7.137.60:38579)
I20260619 02:45:03.310160 30139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f5d2d156ac8748718be69c6877edc88f"
I20260619 02:45:03.310292 30139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:03.311666 30139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69d72ae85ca4e40b41a44fd5d29b319 in term 1.
W20260619 02:45:03.312142 30196 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:03.312209 30196 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:03.311921 29963 leader_election.cc:302] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e69d72ae85ca4e40b41a44fd5d29b319, f5d2d156ac8748718be69c6877edc88f; no voters:
I20260619 02:45:03.312649 30185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:03.312808 30185 raft_consensus.cc:686] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [term 1 LEADER]: Becoming Leader. State: Replica: e69d72ae85ca4e40b41a44fd5d29b319, State: Running, Role: LEADER
I20260619 02:45:03.312925 30185 consensus_queue.cc:237] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.313304 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e69d72ae85ca4e40b41a44fd5d29b319. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.313371 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:03.313951 29947 master_runner.cc:186] Error getting master registration for 127.7.137.61:42569: OK, instance_id { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" instance_seqno: 1781837103098453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:45:03.316284 29947 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:45:03.321563 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.311474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.7.137.61:42569)
0619 02:45:03.311525 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.311528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.312378 (+ 850us) server_negotiation.cc:407] Connection header received
0619 02:45:03.312493 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.312494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.312506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.312537 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.313467 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.314321 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.315529 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.315885 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.315930 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.317805 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.317807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.317807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.317815 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.320121 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.320566 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.320567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.320568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.320622 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.320879 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.320880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.320880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.321027 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.321104 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.321347 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.321437 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:03.322830 30206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:03.323053 30206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:03.325234 30139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [term 1 FOLLOWER]: Refusing update from remote peer e69d72ae85ca4e40b41a44fd5d29b319: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:03.325834 30185 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:03.327490 30171 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e69d72ae85ca4e40b41a44fd5d29b319. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.327592 30171 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.329561 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.311394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.7.137.60:38579)
0619 02:45:03.311788 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.311791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.321384 (+ 9593us) server_negotiation.cc:407] Connection header received
0619 02:45:03.321445 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.321446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.321458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.321486 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.322152 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.322897 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.323907 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.324200 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.324240 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.325775 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.325776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.325777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.325784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.327399 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.328588 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.328589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.328589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.328653 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.328902 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.328903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.328904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.329039 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.329108 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.329316 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.329473 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374}
I20260619 02:45:03.331830 30090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } attempt: 3
I20260619 02:45:03.333401 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.333493 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:03.333940 30171 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.334025 30171 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.334230 30208 mvcc.cc:205] Tried to move back new op lower bound from 7298404775215861760 to 7298404775170396160. Current Snapshot: MvccSnapshot[applied={T|T < 7298404775215861760}]
I20260619 02:45:03.334393 30209 mvcc.cc:205] Tried to move back new op lower bound from 7298404775215861760 to 7298404775170396160. Current Snapshot: MvccSnapshot[applied={T|T < 7298404775215861760}]
I20260619 02:45:03.334859 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.316953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.7.137.62:37165)
0619 02:45:03.317003 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.317004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.321976 (+ 4972us) server_negotiation.cc:407] Connection header received
0619 02:45:03.322082 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.322083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.322094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.322127 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.323774 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.324478 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.330054 (+ 5576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.330370 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.330411 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.331834 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.331835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.331836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.331842 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.333574 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.333971 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.333972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.333972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.334026 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.334250 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.334251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.334252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.334394 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.334485 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.334697 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.334771 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 02:45:03 dist-test-slave-pdnb krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837103, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:03.340581 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.332079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:38579 (local address 127.0.0.1:41900)
0619 02:45:03.332126 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:03.332135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:03.332171 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:03.332387 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:03.332388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:03.332554 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:03.332596 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.332598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.333611 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.333612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.334434 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:03.334436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.335238 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.335239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:03.335320 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.335701 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:03.335705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:03.337221 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:03.339485 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.339486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.339486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.339698 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.339901 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:03.339901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:03.339902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:03.339956 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:03.340250 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:03.340251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:03.340313 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 02:45:03.340380 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:45:03.340476 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:45:03.340584 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.332101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.7.137.60:38579)
0619 02:45:03.332144 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.332146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.332157 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:03.332242 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.332243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.332252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.332295 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.332677 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.333419 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.334630 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.334934 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.334992 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.337313 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:03.337314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:03.337315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:03.337342 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:03.338941 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.339781 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.339782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.339782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.339835 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:03.340028 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:03.340029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:03.340029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:03.340189 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:03.340250 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.340416 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.340493 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:03.341557 30206 catalog_manager.cc:1347] Generated new cluster ID: cb7debe0fd2b4dc19a12c2e0ba631bde
I20260619 02:45:03.341606 30206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:03.342114 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: Bootstrap starting.
I20260619 02:45:03.342854 30090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:03.343139 30090 log.cc:824] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:03.344816 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: No bootstrap required, opened a new log
I20260619 02:45:03.346866 30090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.347002 30090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672d727ac43146aebe34daefacb4d9c7, State: Initialized, Role: FOLLOWER
I20260619 02:45:03.347084 30090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } }
I20260619 02:45:03.347374 30215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.347437 30064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "672d727ac43146aebe34daefacb4d9c7"
I20260619 02:45:03.347710 30215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.347406 30090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:03.347462 30063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:03.348682 30063 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [term 1 FOLLOWER]: Refusing update from remote peer e69d72ae85ca4e40b41a44fd5d29b319: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:03.349143 30212 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [LEADER]: Connected to new peer: Peer: permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:03.349551 30064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69d72ae85ca4e40b41a44fd5d29b319 in term 1.
I20260619 02:45:03.349725 30206 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:03.350621 30206 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:03.351591 30215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e69d72ae85ca4e40b41a44fd5d29b319. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.351675 30215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.351740 30218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.351791 30218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:03.354521 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69d72ae85ca4e40b41a44fd5d29b319" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 37165 } } peers { permanent_uuid: "672d727ac43146aebe34daefacb4d9c7" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 42569 } } peers { permanent_uuid: "f5d2d156ac8748718be69c6877edc88f" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 38579 } } }
I20260619 02:45:03.354611 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319 [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:03.356820 30206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e69d72ae85ca4e40b41a44fd5d29b319: Generated new TSK 0
I20260619 02:45:03.357026 30206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:03.357810 30231 catalog_manager.cc:1259] Loaded cluster ID: cb7debe0fd2b4dc19a12c2e0ba631bde
I20260619 02:45:03.357895 30231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: loading cluster ID for follower catalog manager: success
I20260619 02:45:03.359154 30231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: acquiring CA information for follower catalog manager: success
I20260619 02:45:03.359586 30231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 672d727ac43146aebe34daefacb4d9c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:45:03.390640 30018 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.389241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.7.137.62:37165)
0619 02:45:03.389276 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.389278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.389986 (+ 708us) server_negotiation.cc:407] Connection header received
0619 02:45:03.390093 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.390094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.390105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.390136 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.390567 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44576: BlockingRecv error: recv got EOF from 127.0.0.1:44576 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 02:45:03.391417 30094 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.389037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.7.137.61:42569)
0619 02:45:03.389081 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.389083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.390917 (+ 1834us) server_negotiation.cc:407] Connection header received
0619 02:45:03.391017 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.391018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.391028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.391057 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.391344 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59728: BlockingRecv error: recv got EOF from 127.0.0.1:59728 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:45:03.392102 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.389712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.7.137.60:38579)
0619 02:45:03.389750 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.389752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.391620 (+ 1868us) server_negotiation.cc:407] Connection header received
0619 02:45:03.391714 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.391715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.391724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.391753 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.392043 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41906: BlockingRecv error: recv got EOF from 127.0.0.1:41906 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:03.397374 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.393706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.7.137.61:42569)
0619 02:45:03.393739 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.393740 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.393932 (+ 192us) server_negotiation.cc:407] Connection header received
0619 02:45:03.394032 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.394033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.394070 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.394115 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.394302 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.395085 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.396222 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.396517 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.396556 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.396988 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.397120 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.397171 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.397285 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.397305 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:45:03.400519 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.399644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.7.137.60:38579)
0619 02:45:03.399681 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.399683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.400014 (+ 331us) server_negotiation.cc:407] Connection header received
0619 02:45:03.400115 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.400116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.400133 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.400161 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.400470 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41916: BlockingRecv error: recv got EOF from 127.0.0.1:41916 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:03.405869 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.402699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.7.137.61:42569)
0619 02:45:03.402741 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.402743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.403017 (+ 274us) server_negotiation.cc:407] Connection header received
0619 02:45:03.403105 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.403106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.403114 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.403140 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.403298 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.404052 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.405128 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.405418 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.405454 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.405626 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.405656 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.405681 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.405773 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.405789 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 02:45:03.410013 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.409248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.7.137.60:38579)
0619 02:45:03.409289 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.409291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.409505 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:45:03.409597 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.409598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.409609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.409636 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.409964 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41932: BlockingRecv error: recv got EOF from 127.0.0.1:41932 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:03.419781 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.413057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.7.137.61:42569)
0619 02:45:03.413101 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.413103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.416911 (+ 3808us) server_negotiation.cc:407] Connection header received
0619 02:45:03.417018 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.417019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.417028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.417054 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.417216 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.417943 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.419028 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.419316 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.419355 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.419538 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.419570 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.419597 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.419697 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.419714 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:45:03.422921 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.422094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.7.137.60:38579)
0619 02:45:03.422142 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.422144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.422358 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:45:03.422454 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.422455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.422466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.422494 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.422822 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41934: BlockingRecv error: recv got EOF from 127.0.0.1:41934 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:03.431330 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.428108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.7.137.61:42569)
0619 02:45:03.428150 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.428152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.428370 (+ 218us) server_negotiation.cc:407] Connection header received
0619 02:45:03.428469 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.428470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.428480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.428508 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.428683 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.429440 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.430529 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.430803 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.430862 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.431070 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.431103 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.431131 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.431237 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.431256 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:45:03.434407 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.433640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.7.137.60:38579)
0619 02:45:03.433680 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.433682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.433871 (+ 189us) server_negotiation.cc:407] Connection header received
0619 02:45:03.433969 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.433970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.433981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.434010 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.434351 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41946: BlockingRecv error: recv got EOF from 127.0.0.1:41946 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:03.446762 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.443573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.7.137.61:42569)
0619 02:45:03.443613 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.443615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.443834 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:45:03.443956 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.443957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.443967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.443996 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.444178 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.444934 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.445972 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.446253 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.446297 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.446498 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.446531 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.446560 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.446667 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.446686 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:45:03.449867 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.449076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.7.137.60:38579)
0619 02:45:03.449116 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.449118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.449295 (+ 177us) server_negotiation.cc:407] Connection header received
0619 02:45:03.449409 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.449410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.449422 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.449452 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.449799 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41954: BlockingRecv error: recv got EOF from 127.0.0.1:41954 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:03.470589 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.467234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.7.137.61:42569)
0619 02:45:03.467275 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.467277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.467556 (+ 279us) server_negotiation.cc:407] Connection header received
0619 02:45:03.467668 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.467669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.467680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.467710 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.467918 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.468635 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.469792 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.470077 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.470122 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.470320 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.470353 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.470380 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.470488 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.470507 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:45:03.473664 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.472923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.7.137.60:38579)
0619 02:45:03.472961 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.472963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.473135 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:45:03.473230 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.473231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.473243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.473271 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.473610 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41960: BlockingRecv error: recv got EOF from 127.0.0.1:41960 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:03.509796 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.506934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.7.137.61:42569)
0619 02:45:03.506985 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.506988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.507151 (+ 163us) server_negotiation.cc:407] Connection header received
0619 02:45:03.507254 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.507255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.507267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.507322 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.507448 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.508091 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.509087 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.509294 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.509366 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.509512 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.509555 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.509618 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.509680 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.509698 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 02:45:03.512074 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.511578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.7.137.60:38579)
0619 02:45:03.511628 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.511630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.511637 (+ 7us) server_negotiation.cc:407] Connection header received
0619 02:45:03.511661 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.511662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.511672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.511723 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.512020 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41976: BlockingRecv error: recv got EOF from 127.0.0.1:41976 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:03.579802 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.577548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.7.137.61:42569)
0619 02:45:03.577620 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.577622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.577665 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:45:03.577716 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.577717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.577726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.577779 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.577891 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.578373 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.579178 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.579365 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.579427 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.579552 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.579580 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.579619 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.579669 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.579685 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 02:45:03.582161 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.581546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.7.137.60:38579)
0619 02:45:03.581604 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.581606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.581615 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:03.581715 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.581716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.581726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.581775 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.582097 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41984: BlockingRecv error: recv got EOF from 127.0.0.1:41984 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:45:03.713457 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.711273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.7.137.61:42569)
0619 02:45:03.711361 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.711363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.711390 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:45:03.711462 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.711463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.711472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.711525 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.711646 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.712138 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.712869 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.713060 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.713114 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.713225 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.713253 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.713291 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.713338 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.713355 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 02:45:03.715613 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.715105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.7.137.60:38579)
0619 02:45:03.715173 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.715175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.715184 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:03.715232 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.715233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.715247 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.715289 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.715569 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42000: BlockingRecv error: recv got EOF from 127.0.0.1:42000 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:45:03.974841 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:03.972767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.7.137.61:42569)
0619 02:45:03.972818 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.972820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.972891 (+ 71us) server_negotiation.cc:407] Connection header received
0619 02:45:03.972948 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.972949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.972958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.972998 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:03.973132 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.973604 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.974298 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:03.974470 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:03.974519 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:03.974626 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:03.974651 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:03.974688 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:03.974735 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:45:03.974749 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:45:03.976874 30169 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:03.976385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.7.137.60:38579)
0619 02:45:03.976426 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:03.976428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:03.976437 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:03.976476 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:03.976476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:03.976485 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:03.976525 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:03.976814 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42002: BlockingRecv error: recv got EOF from 127.0.0.1:42002 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 02:45:04.304325 30182 catalog_manager.cc:1259] Loaded cluster ID: cb7debe0fd2b4dc19a12c2e0ba631bde
I20260619 02:45:04.304400 30182 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: loading cluster ID for follower catalog manager: success
I20260619 02:45:04.305289 30182 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: acquiring CA information for follower catalog manager: success
I20260619 02:45:04.305564 30182 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5d2d156ac8748718be69c6877edc88f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:45:04.492158 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:04.489924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.7.137.61:42569)
0619 02:45:04.490050 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:04.490051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:04.490081 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:45:04.490152 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:04.490152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:04.490162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:04.490224 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:04.490355 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.490835 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.491536 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.491716 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.491762 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:04.491881 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:04.491906 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:04.491944 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:04.491995 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 02:45:04.492024 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260619 02:45:04.496001 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:04.493832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.7.137.60:38579)
0619 02:45:04.493946 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:04.493948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:04.493957 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:04.494024 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:04.494024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:04.494072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:04.494141 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:04.494254 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.494719 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.495379 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.495570 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.495617 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:04.495737 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:04.495838 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:04.495895 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:04.495924 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 02:45:04.495940 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260619 02:45:04.500499 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:04.497786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.7.137.60:38579)
0619 02:45:04.497822 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:04.497824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:04.497932 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:45:04.498014 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:04.498015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:04.498025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:04.498065 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:04.498201 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.498917 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.499780 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.500068 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.500129 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:04.500254 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:04.500285 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:04.500336 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:04.500405 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:45:04.500428 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:04.500998 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:04.497588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.7.137.62:37165)
0619 02:45:04.497690 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:04.497691 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:04.498034 (+ 343us) server_negotiation.cc:407] Connection header received
0619 02:45:04.498085 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:04.498086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:04.498122 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:04.498211 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:04.498338 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.498813 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.499904 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.500222 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.500282 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:04.500524 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:04.500642 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:04.500700 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:04.500892 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 02:45:04.500920 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":65,"threads_started":1}
I20260619 02:45:04.501060 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:04.497581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.7.137.61:42569)
0619 02:45:04.497622 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:04.497624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:04.497727 (+ 103us) server_negotiation.cc:407] Connection header received
0619 02:45:04.497835 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:04.497836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:04.497847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:04.497913 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:04.498290 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.498981 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.500042 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:04.500360 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:04.500418 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:04.500692 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:04.500751 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:04.500780 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:04.500950 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 02:45:04.500974 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:04.502755 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 29947
I20260619 02:45:04.508272 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 30022
I20260619 02:45:04.512956 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 30097
2026-06-19T02:45:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2857 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:45:04 dist-test-slave-pdnb krb5kdc[30364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:45:04 dist-test-slave-pdnb krb5kdc[30364](info): set up 2 sockets
Jun 19 02:45:04 dist-test-slave-pdnb krb5kdc[30364](info): commencing operation
krb5kdc: starting...
W20260619 02:45:05.541481 7716 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/test-user.keytab.
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.137.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
Entry for principal kudu/127.7.137.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab.
I20260619 02:45:05.653625 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36017
--webserver_interface=127.7.137.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:45:05.732198 30377 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:05.732391 30377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:05.732409 30377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:05.732427 30377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:05.733546 30377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:05.733585 30377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:05.733600 30377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:05.733613 30377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:05.733624 30377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:05.734978 30377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.62.keytab
--principal=kudu/127.7.137.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.62:36017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.137.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:05.735152 30377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:05.735301 30377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:05.740283 30382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:05.740420 30377 server_base.cc:1034] running on GCE node
W20260619 02:45:05.740283 30385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:05.740280 30383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:05.740779 30377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:05.741119 30377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:05.742293 30377 hybrid_clock.cc:648] HybridClock initialized: now 1781837105742269 us; error 38 us; skew 500 ppm
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:05.744832 30377 init.cc:377] Logged in from keytab as kudu/127.7.137.62@KRBTEST.COM (short username kudu)
I20260619 02:45:05.745291 30377 webserver.cc:460] Webserver started at http://127.7.137.62:35693/ using document root <none> and password file <none>
I20260619 02:45:05.745528 30377 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:05.745594 30377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:05.745726 30377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:05.746750 30377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data/instance:
uuid: "3f9553ccf09e46abb6eab5c0a06ec10c"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "a63e4e8c0c86a22f73168298cc9ef4fd"
server_key_iv: "4a46b982f31fa6c3bcaf62f7a5b120db"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.747109 30377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance:
uuid: "3f9553ccf09e46abb6eab5c0a06ec10c"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "a63e4e8c0c86a22f73168298cc9ef4fd"
server_key_iv: "4a46b982f31fa6c3bcaf62f7a5b120db"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.748440 30377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 02:45:05.749243 30392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:05.749425 30377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:05.749519 30377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal
uuid: "3f9553ccf09e46abb6eab5c0a06ec10c"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "a63e4e8c0c86a22f73168298cc9ef4fd"
server_key_iv: "4a46b982f31fa6c3bcaf62f7a5b120db"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.749605 30377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:05.762188 30377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:05.763963 30377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:05.764112 30377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:05.768013 30377 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.62:36017
I20260619 02:45:05.768062 30444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.62:36017 every 8 connection(s)
I20260619 02:45:05.768395 30377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/data/info.pb
I20260619 02:45:05.769470 30445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:05.770942 30445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:05.778060 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 30377
I20260619 02:45:05.778174 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-0/wal/instance
I20260619 02:45:05.778373 7716 external_mini_cluster.cc:1428] Setting key 8c1464a626ac8805593ca8b2e6b4ded7
I20260619 02:45:05.780946 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.771550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36017 (local address 127.0.0.1:39392)
0619 02:45:05.771695 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.771727 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:05.772153 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:05.772395 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:05.772397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:05.772621 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:05.772825 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.772829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.773797 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.773798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.774464 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.774467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.774929 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.774930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.774988 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.775516 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:05.775525 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:05.776904 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:05.779013 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.779021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.779024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.779342 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.779655 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.779656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.779656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.779745 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.780298 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:05.780304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:05.780396 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 02:45:05.780510 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:45:05.780693 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260619 02:45:05.781176 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.771688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.7.137.62:36017)
0619 02:45:05.771803 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:05.771807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:05.771839 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:45:05.772274 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:05.772275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:05.772290 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:05.772362 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:05.772902 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.773688 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.774552 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.774838 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.774913 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.777000 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:05.777006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:05.777007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:05.777013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:05.778891 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.779448 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.779452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.779453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.779580 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.779819 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.779820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.779820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.780218 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:05.780308 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:05.780989 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 02:45:05.781071 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260619 02:45:05.781855 30445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:05.782222 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.782097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33480)
0619 02:45:05.782151 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.782198 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 02:45:05.782410 30393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
I20260619 02:45:05.782835 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.782748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33496)
0619 02:45:05.782794 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.782818 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 02:45:05.782953 30445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:33853: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.137.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
Entry for principal kudu/127.7.137.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab.
I20260619 02:45:05.803287 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:33853
--webserver_interface=127.7.137.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:05.832166 30445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } attempt: 1
I20260619 02:45:05.832682 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.832550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33512)
0619 02:45:05.832616 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.832652 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:45:05.833040 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.832947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33520)
0619 02:45:05.832995 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.833024 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 02:45:05.833123 30445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:33853: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
W20260619 02:45:05.883813 30452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:05.884073 30452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:05.884104 30452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:05.884145 30452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:05.885376 30452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:05.885449 30452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:05.885473 30452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:05.885514 30452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:05.885538 30452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:05.887048 30452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.61.keytab
--principal=kudu/127.7.137.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.61:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.137.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:05.887332 30452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:05.887573 30452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:45:05.892426 30452 server_base.cc:1034] running on GCE node
W20260619 02:45:05.892393 30460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:05.892383 30458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:05.892398 30457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:05.892892 30452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:05.893234 30452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:05.894399 30452 hybrid_clock.cc:648] HybridClock initialized: now 1781837105894378 us; error 36 us; skew 500 ppm
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:05.897042 30452 init.cc:377] Logged in from keytab as kudu/127.7.137.61@KRBTEST.COM (short username kudu)
I20260619 02:45:05.897565 30452 webserver.cc:460] Webserver started at http://127.7.137.61:38387/ using document root <none> and password file <none>
I20260619 02:45:05.897807 30452 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:05.897881 30452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:05.898039 30452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:05.899116 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data/instance:
uuid: "02f5f7d884b44b098f0a2748d289efe1"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "e735280c7cebc8a319f2adbc0201930d"
server_key_iv: "80ba9f48a99f4829ae118cd8335ffbc9"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.899487 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal/instance:
uuid: "02f5f7d884b44b098f0a2748d289efe1"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "e735280c7cebc8a319f2adbc0201930d"
server_key_iv: "80ba9f48a99f4829ae118cd8335ffbc9"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.900876 30452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 02:45:05.901258 30445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } attempt: 2
I20260619 02:45:05.901657 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.901527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33524)
0619 02:45:05.901586 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.901618 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:45:05.901690 30467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:05.901873 30452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:45:05.901978 30452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal
uuid: "02f5f7d884b44b098f0a2748d289efe1"
format_stamp: "Formatted at 2026-06-19 02:45:05 on dist-test-slave-pdnb"
server_key: "e735280c7cebc8a319f2adbc0201930d"
server_key_iv: "80ba9f48a99f4829ae118cd8335ffbc9"
server_key_version: "encryptionkey@0"
I20260619 02:45:05.902091 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.901994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33530)
0619 02:45:05.902032 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.902067 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260619 02:45:05.902051 30452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260619 02:45:05.902223 30445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.61:33853: Network error: Client connection negotiation failed: client connection to 127.7.137.61:33853: connect: Connection refused (error 111)
I20260619 02:45:05.913034 30452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:05.914742 30452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:05.914911 30452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:05.919171 30452 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.61:33853
I20260619 02:45:05.919202 30519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.61:33853 every 8 connection(s)
I20260619 02:45:05.919593 30452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/data/info.pb
I20260619 02:45:05.920562 30520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:05.922099 30520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:05.927796 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 30452
I20260619 02:45:05.927932 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-1/wal/instance
I20260619 02:45:05.928133 7716 external_mini_cluster.cc:1428] Setting key cd1f022656c1e28933d88796282bb927
I20260619 02:45:05.931491 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.922744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36017 (local address 127.0.0.1:39402)
0619 02:45:05.922965 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.923000 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:05.923456 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:05.923731 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:05.923734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:05.923994 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:05.924182 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.924186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.924931 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.924932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.925745 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.925747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.926145 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.926146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.926257 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.926819 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:05.926828 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:05.928255 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:05.930056 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.930063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.930065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.930378 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.930569 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.930570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.930571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.930707 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.931013 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:05.931018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:05.931131 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:45:05.931216 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:45:05.931337 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260619 02:45:05.931509 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.922772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.7.137.62:36017)
0619 02:45:05.922839 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:05.922842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:05.923087 (+ 245us) server_negotiation.cc:407] Connection header received
0619 02:45:05.923575 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:05.923576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:05.923589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:05.923654 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:05.924268 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.924835 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.925851 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.926058 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.926116 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.928350 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:05.928352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:05.928353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:05.928361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:05.929969 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.930454 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.930454 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.930455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.930505 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.930788 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.930789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.930790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.930936 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:05.931008 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:05.931263 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:45:05.931358 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:45:05.932171 30520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.137.60@KRBTEST.COM; defaulting to no policy
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.137.60@KRBTEST.COM" created.
I20260619 02:45:05.939941 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.932462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33532)
0619 02:45:05.932520 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.932533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:05.932579 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:05.932831 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:05.932832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:05.932989 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:05.933049 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.933051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.933863 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.933864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.934715 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.934717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.935182 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.935184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.935264 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.935610 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:05.935615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:05.936918 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:05.938703 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.938704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.938704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.938863 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.939120 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.939120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.939121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.939173 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.939584 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:05.939585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:05.939624 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:45:05.939711 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:45:05.939808 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:45:05.940016 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.932516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.7.137.61:33853)
0619 02:45:05.932660 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:05.932665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:05.932686 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:45:05.932708 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:05.932710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:05.932741 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:05.932821 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:05.933129 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.933777 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.934800 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.935102 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.935159 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.937017 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:05.937024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:05.937024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:05.937030 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:05.938544 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.938956 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.938959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.938959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.939053 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.939280 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.939281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.939281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.939472 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:05.939594 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:05.939741 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:45:05.939877 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260619 02:45:05.941121 30520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:05.941469 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.941343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48490)
0619 02:45:05.941393 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.941440 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 02:45:05.941635 30470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
I20260619 02:45:05.941998 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.941931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48504)
0619 02:45:05.941954 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.941976 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 02:45:05.942128 30520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:39321: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
Entry for principal kudu/127.7.137.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab.
I20260619 02:45:05.954808 7716 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_8LRmf/build/release/bin/kudu
/tmp/dist-test-task_8LRmf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:39321
--webserver_interface=127.7.137.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.137.20:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:45:05.991362 30520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } attempt: 1
I20260619 02:45:05.991894 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.991752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48510)
0619 02:45:05.991821 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.991865 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 02:45:05.992256 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.992176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48524)
0619 02:45:05.992216 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.992240 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:45:05.992353 30520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:39321: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
I20260619 02:45:05.993378 30445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } attempt: 3
Jun 19 02:45:05 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:45:06.000415 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.993727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.7.137.61:33853)
0619 02:45:05.993768 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:05.993771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:05.993782 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:45:05.993875 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:05.993876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:05.993889 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:05.993957 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:05.994270 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.994764 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.995536 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.995753 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.995831 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.997962 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:05.997964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:05.997964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:05.997970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:05.999339 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.999746 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.999747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.999747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:05.999794 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:05.999993 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:05.999993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:05.999993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.000089 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.000140 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.000293 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.000354 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 02:45:06.000478 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:05.993676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33542)
0619 02:45:05.993732 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:05.993743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:05.993797 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:05.993965 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:05.993966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:05.994127 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:05.994190 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.994192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.994853 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.994854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.995415 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:05.995417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:05.995828 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:05.995829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:05.995915 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:05.996370 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:05.996383 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:05.997881 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:05.999432 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.999433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.999434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.999667 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:05.999857 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:05.999858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:05.999858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:05.999924 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.000146 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.000148 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.000200 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.000269 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.000371 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 02:45:06.000985 30445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:06.001305 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.001208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48536)
0619 02:45:06.001254 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.001283 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 02:45:06.001610 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.001535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48544)
0619 02:45:06.001573 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.001594 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 02:45:06.001708 30445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:39321: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
I20260619 02:45:06.032823 30445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } attempt: 1
I20260619 02:45:06.033278 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.033151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48548)
0619 02:45:06.033213 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.033251 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:45:06.033649 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.033571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48554)
0619 02:45:06.033610 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.033633 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:45:06.033754 30445 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:39321: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
W20260619 02:45:06.038774 30527 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:45:06.038968 30527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:45:06.038986 30527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:45:06.039033 30527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:45:06.040145 30527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:45:06.040184 30527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:45:06.040198 30527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:45:06.040236 30527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:45:06.040263 30527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:45:06.041782 30527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.137.20:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.137.62:36017,127.7.137.61:33853,127.7.137.60:39321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/krb5kdc/kudu_127.7.137.60.keytab
--principal=kudu/127.7.137.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.137.60:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.137.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 01:55:14 UTC on ccycloud.quasar-liqyol.root.comops.site
build id 4454980
I20260619 02:45:06.042029 30527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:45:06.042261 30527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:45:06.046898 30535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:45:06.046978 30533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:06.047127 30527 server_base.cc:1034] running on GCE node
W20260619 02:45:06.046908 30532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:45:06.047431 30527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:45:06.047706 30527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:45:06.048839 30527 hybrid_clock.cc:648] HybridClock initialized: now 1781837106048826 us; error 28 us; skew 500 ppm
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837106, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:45:06.051358 30527 init.cc:377] Logged in from keytab as kudu/127.7.137.60@KRBTEST.COM (short username kudu)
I20260619 02:45:06.051836 30527 webserver.cc:460] Webserver started at http://127.7.137.60:36703/ using document root <none> and password file <none>
I20260619 02:45:06.052049 30527 fs_manager.cc:359] Metadata directory not provided
I20260619 02:45:06.052091 30527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:45:06.052175 30527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:45:06.053299 30527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data/instance:
uuid: "d6c683b91a5a44658460f837e100677e"
format_stamp: "Formatted at 2026-06-19 02:45:06 on dist-test-slave-pdnb"
server_key: "3d4aa77955256300973c179304eb27ad"
server_key_iv: "e749f1ce32d5cb1bbd8ba1ae85482585"
server_key_version: "encryptionkey@0"
I20260619 02:45:06.053606 30527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal/instance:
uuid: "d6c683b91a5a44658460f837e100677e"
format_stamp: "Formatted at 2026-06-19 02:45:06 on dist-test-slave-pdnb"
server_key: "3d4aa77955256300973c179304eb27ad"
server_key_iv: "e749f1ce32d5cb1bbd8ba1ae85482585"
server_key_version: "encryptionkey@0"
I20260619 02:45:06.054824 30527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:45:06.055506 30542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:45:06.055677 30527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 02:45:06.055773 30527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data,/tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal
uuid: "d6c683b91a5a44658460f837e100677e"
format_stamp: "Formatted at 2026-06-19 02:45:06 on dist-test-slave-pdnb"
server_key: "3d4aa77955256300973c179304eb27ad"
server_key_iv: "e749f1ce32d5cb1bbd8ba1ae85482585"
server_key_version: "encryptionkey@0"
I20260619 02:45:06.055855 30527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:45:06.060477 30520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } attempt: 2
I20260619 02:45:06.060858 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.060756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48556)
0619 02:45:06.060796 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.060830 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 02:45:06.061182 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.061094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48572)
0619 02:45:06.061136 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.061159 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 02:45:06.061262 30520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.137.60:39321: Network error: Client connection negotiation failed: client connection to 127.7.137.60:39321: connect: Connection refused (error 111)
I20260619 02:45:06.070888 30527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:45:06.073146 30527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:45:06.073305 30527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:45:06.077196 30527 rpc_server.cc:307] RPC server started. Bound to: 127.7.137.60:39321
I20260619 02:45:06.077226 30594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.137.60:39321 every 8 connection(s)
I20260619 02:45:06.077572 30527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/data/info.pb
I20260619 02:45:06.078505 30595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:45:06.079370 7716 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_8LRmf/build/release/bin/kudu as pid 30527
I20260619 02:45:06.079473 7716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_8LRmf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836839601721-7716-0/minicluster-data/master-2/wal/instance
I20260619 02:45:06.079691 7716 external_mini_cluster.cc:1428] Setting key 17608d537f0f492abd163db92ec10d87
I20260619 02:45:06.080587 30595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837106, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:06.089506 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.081572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.62:36017 (local address 127.0.0.1:39406)
0619 02:45:06.081717 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.081753 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:06.081816 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:06.081961 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:06.081964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:06.082296 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:06.082408 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.082414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.083027 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.083028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.083755 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.083756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.084113 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.084114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.084193 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.084806 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:06.084814 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:06.086284 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:06.087813 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.087820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.087822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.088129 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.088374 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.088375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.088375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.088483 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.088789 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.088794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.088952 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.089062 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.089242 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260619 02:45:06.089915 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.081659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.7.137.62:36017)
0619 02:45:06.081710 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.081712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.081819 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:45:06.081862 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.081863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.081875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.081940 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.082502 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.082944 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.083835 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.084018 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.084071 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.086368 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.086370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.086370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.086378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.087751 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.088243 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.088244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.088244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.088309 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.088558 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.088559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.088559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.088714 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.088800 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.089483 (+ 683us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.089820 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 02:45:06.089926 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.080246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.7.137.60:39321)
0619 02:45:06.080422 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.080431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.080464 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:45:06.080933 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.080940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.080994 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.081105 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.081906 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.083004 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.083715 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.084091 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.084193 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.086967 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.086973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.086974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.086979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.088572 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.088973 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.088975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.088976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.089021 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.089239 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.089240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.089240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.089423 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.089717 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.089759 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.089827 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260619 02:45:06.090404 30595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837106, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.62@KRBTEST.COM
I20260619 02:45:06.097313 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.090658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.61:33853 (local address 127.0.0.1:33554)
0619 02:45:06.090741 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.090751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:06.090786 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:06.090915 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:06.090916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:06.091065 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:06.091113 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.091115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.091843 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.091844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.092388 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.092390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.092847 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.092848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.092915 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.093313 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:06.093317 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:06.094448 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:06.096316 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.096317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.096318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.096494 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.096686 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.096687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.096687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.096772 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.097019 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.097020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.097060 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.097139 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.097224 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:45:06.097319 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.090746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.7.137.61:33853)
0619 02:45:06.090797 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.090799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.090808 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:45:06.090833 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.090835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.090846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.090902 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.091187 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.091775 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.092458 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.092783 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.092841 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.094537 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.094539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.094539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.094544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.096218 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.096564 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.096565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.096565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.096621 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.096838 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.096839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.096839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.096947 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.097026 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.097164 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.097232 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:06.097633 30595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:45:06.098219 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.090698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.7.137.62:36017)
0619 02:45:06.090733 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.090735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.090786 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:45:06.090903 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.090904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.090915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.090982 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.091490 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.091973 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.092687 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.093004 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.093069 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.095155 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.095157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.095157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.095166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.096584 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.096972 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.096973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.096973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.097026 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.097217 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.097218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.097218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.097320 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.097683 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.098044 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.098128 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837106, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.60@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.137.61@KRBTEST.COM
I20260619 02:45:06.104049 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.097865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48578)
0619 02:45:06.097960 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.097968 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:06.098032 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:06.098157 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:06.098158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:06.098258 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:06.098282 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.098283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.099097 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.099097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.099685 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.099687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.100016 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.100017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.100072 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.100431 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:06.100435 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:06.101614 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:06.103065 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.103066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.103067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.103251 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.103443 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.103443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.103443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.103502 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.103725 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.103726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.103757 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.103827 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.103925 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 02:45:06.104130 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.097924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.7.137.60:39321)
0619 02:45:06.097959 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.097961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.098016 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:45:06.098091 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.098091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.098100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.098149 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.098365 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.099024 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.099757 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.099945 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.100004 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.101689 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.101690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.101690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.101697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.102986 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.103317 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.103318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.103319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.103374 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.103564 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.103565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.103565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.103674 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.103721 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.103850 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.103974 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":19}
I20260619 02:45:06.105551 30595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: Bootstrap starting.
I20260619 02:45:06.106320 30595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:06.106600 30595 log.cc:824] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:06.106765 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.098800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.7.137.61:33853)
0619 02:45:06.098836 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.098838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.098879 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:45:06.098932 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.098932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.098940 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.098996 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.099384 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.099960 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.100792 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.101028 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.101079 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.103749 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.103749 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.103750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.103755 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.105505 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.105888 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.105889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.105889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.105939 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.106150 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.106151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.106151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.106257 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.106406 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.106519 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.106590 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 02:45:06.107513 30595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: No bootstrap required, opened a new log
I20260619 02:45:06.108098 7716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:45:06.109015 30445 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } attempt: 2
I20260619 02:45:06.109216 30595 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.109364 30595 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c683b91a5a44658460f837e100677e, State: Initialized, Role: FOLLOWER
I20260619 02:45:06.109449 30595 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.109741 30601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.109807 30595 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:06.108880 30419 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39422
I20260619 02:45:06.109877 30601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:45:06.111644 30612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:06.111693 30612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.62@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:06.112426 30527 master_runner.cc:186] Error getting master registration for 127.7.137.62:36017: OK, instance_id { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" instance_seqno: 1781837105766890 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:45:06.114900 30527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:45:06.116840 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.109340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.7.137.60:39321)
0619 02:45:06.109385 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.109387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.109395 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:06.109415 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.109416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.109424 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.109480 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.109834 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.110524 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.111497 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.111767 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.111840 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.113858 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.113859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.113859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.113866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.115627 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.116006 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.116007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.116008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.116066 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.116259 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.116260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.116260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.116409 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.116480 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.116629 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.116702 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 02:45:06.116844 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.109242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48588)
0619 02:45:06.109327 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.109338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:06.109372 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:06.109534 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:06.109536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:06.109698 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:06.109751 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.109753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.110624 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.110625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.111372 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.111375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.111846 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.111848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.111906 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.112280 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:06.112284 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:06.113761 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:06.115721 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.115722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.115723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.115933 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.116136 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.116136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.116137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.116196 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.116482 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.116483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.116527 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.116592 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.116676 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 02:45:06.118284 30445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: Bootstrap starting.
I20260619 02:45:06.118963 30445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:06.119277 30445 log.cc:824] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:06.120101 30445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: No bootstrap required, opened a new log
I20260619 02:45:06.121455 30445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.121598 30445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9553ccf09e46abb6eab5c0a06ec10c, State: Initialized, Role: FOLLOWER
I20260619 02:45:06.121681 30445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.121963 30445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:06.121979 30419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:45:06.122094 30419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:06.121953 30615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.122268 30615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:06.122848 30419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.123123 30419 leader_election.cc:288] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [CANDIDATE]: Term 1 election: Requested vote from peers 02f5f7d884b44b098f0a2748d289efe1 (127.7.137.61:33853), d6c683b91a5a44658460f837e100677e (127.7.137.60:39321)
I20260619 02:45:06.123628 30569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d6c683b91a5a44658460f837e100677e"
I20260619 02:45:06.123746 30569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:06.124697 30569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f9553ccf09e46abb6eab5c0a06ec10c in term 1.
I20260619 02:45:06.125473 30395 leader_election.cc:302] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9553ccf09e46abb6eab5c0a06ec10c, d6c683b91a5a44658460f837e100677e; no voters:
I20260619 02:45:06.125655 30615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:45:06.125797 30615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [term 1 LEADER]: Becoming Leader. State: Replica: 3f9553ccf09e46abb6eab5c0a06ec10c, State: Running, Role: LEADER
I20260619 02:45:06.125871 30615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.126242 30615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9553ccf09e46abb6eab5c0a06ec10c. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.126319 30615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:06.124698 30377 master_runner.cc:186] Error getting master registration for 127.7.137.61:33853: OK, instance_id { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" instance_seqno: 1781837105918013 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:45:06.126888 30377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 02:45:06.126891 30626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:06.127262 30626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:06.131217 30636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:45:06.131394 30636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:45:06.134943 30569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [term 1 FOLLOWER]: Refusing update from remote peer 3f9553ccf09e46abb6eab5c0a06ec10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:06.135341 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.127021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.7.137.61:33853)
0619 02:45:06.127079 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.127082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.127403 (+ 321us) server_negotiation.cc:407] Connection header received
0619 02:45:06.127494 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.127495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.127505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.127533 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.128233 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.128967 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.130301 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.130606 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.130645 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.132313 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.132314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.132315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.132322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.134076 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.134451 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.134452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.134453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.134503 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.134694 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.134695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.134695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.134813 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.134882 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.135119 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.135245 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 02:45:06.135335 30615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:45:06.137274 30601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9553ccf09e46abb6eab5c0a06ec10c. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.137403 30601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:06.137482 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.124629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.7.137.62:36017)
0619 02:45:06.124667 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.124669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.125428 (+ 759us) server_negotiation.cc:407] Connection header received
0619 02:45:06.125547 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.125548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.125557 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.125591 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.127434 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.128183 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.129173 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.129451 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.129505 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.133397 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.133398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.133398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.133405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.135897 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.136358 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.136359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.136359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.136422 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.136675 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.136676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.136676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.136856 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.136949 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.137280 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.137394 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 02:45:06.138552 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.124812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.7.137.60:39321)
0619 02:45:06.127064 (+ 2252us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.127066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.127074 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:06.127093 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.127094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.127102 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.127163 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.128210 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.128977 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.130407 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.130726 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.130771 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.132655 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.132656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.132656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.132663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.135008 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.137493 (+ 2485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.137494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.137494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.137555 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.137981 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.137982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.137983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.138115 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.138182 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.138383 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.138453 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2233}
I20260619 02:45:06.149804 30615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.149909 30615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: This master's current role is: LEADER
I20260619 02:45:06.150477 30636 catalog_manager.cc:1347] Generated new cluster ID: 964cd740eda04b5cb7cb631aabdb3936
I20260619 02:45:06.150525 30636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:45:06.150715 30601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.150779 30601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:06.151158 30639 mvcc.cc:205] Tried to move back new op lower bound from 7298404786724327424 to 7298404786692108288. Current Snapshot: MvccSnapshot[applied={T|T < 7298404786724327424}]
I20260619 02:45:06.152381 30520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } attempt: 3
Jun 19 02:45:06 dist-test-slave-pdnb krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837105, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.137.61@KRBTEST.COM for kudu/127.7.137.60@KRBTEST.COM
I20260619 02:45:06.165277 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.152626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.137.60:39321 (local address 127.0.0.1:48618)
0619 02:45:06.152669 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 02:45:06.152680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:45:06.152747 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:45:06.157282 (+ 4535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:45:06.157283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:45:06.157476 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:45:06.157521 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.157523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.158426 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.158428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.159203 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:45:06.159205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.159657 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.159658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:45:06.159725 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.160236 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:45:06.160257 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:45:06.162169 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:45:06.164162 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.164163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.164163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.164376 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.164599 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:45:06.164599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:45:06.164600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:45:06.164662 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:45:06.164959 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:45:06.164960 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:45:06.165014 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 02:45:06.165074 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 02:45:06.165174 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 02:45:06.165795 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:06.157031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.7.137.60:39321)
0619 02:45:06.157153 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.157160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.157168 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:06.157187 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.157188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.157198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.157258 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.157582 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.158343 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.159267 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:06.159562 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:06.159631 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:06.162381 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:45:06.162382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:45:06.162382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:45:06.162389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:45:06.164079 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.164461 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.164462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.164463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.164528 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:45:06.164752 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:45:06.164752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:45:06.164753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:45:06.164865 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:45:06.164919 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:06.165636 (+ 717us) server_negotiation.cc:299] Negotiation successful
0619 02:45:06.165709 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 02:45:06.167533 30520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: Bootstrap starting.
I20260619 02:45:06.167693 30636 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:45:06.168242 30520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: Neither blocks nor log segments found. Creating new log.
I20260619 02:45:06.168403 30636 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:45:06.168565 30520 log.cc:824] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:45:06.175999 30520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: No bootstrap required, opened a new log
I20260619 02:45:06.178164 30520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.178308 30520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f5f7d884b44b098f0a2748d289efe1, State: Initialized, Role: FOLLOWER
I20260619 02:45:06.178395 30520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } }
I20260619 02:45:06.178722 30520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:45:06.178774 30494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "02f5f7d884b44b098f0a2748d289efe1"
I20260619 02:45:06.178856 30494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:45:06.179823 30494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f9553ccf09e46abb6eab5c0a06ec10c in term 1.
I20260619 02:45:06.180855 30493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [term 1 FOLLOWER]: Refusing update from remote peer 3f9553ccf09e46abb6eab5c0a06ec10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:45:06.181155 30643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:45:06.182333 30657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:45:06.182382 30657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:45:06.182655 30636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c: Generated new TSK 0
I20260619 02:45:06.182833 30636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:45:06.183027 30646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.183117 30646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:06.184697 30646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9553ccf09e46abb6eab5c0a06ec10c. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.184793 30646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:45:06.184675 30661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.185105 30661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:45:06.212591 30598 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.210892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.7.137.60:39321)
0619 02:45:06.210945 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.210947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.211939 (+ 992us) server_negotiation.cc:407] Connection header received
0619 02:45:06.212049 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.212050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.212061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.212093 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.212524 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48622: BlockingRecv error: recv got EOF from 127.0.0.1:48622 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 02:45:06.213378 30448 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.211298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39436 (local address 127.7.137.62:36017)
0619 02:45:06.211337 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.211339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.212872 (+ 1533us) server_negotiation.cc:407] Connection header received
0619 02:45:06.212971 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.212972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.212983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.213012 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.213312 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39436: BlockingRecv error: recv got EOF from 127.0.0.1:39436 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 02:45:06.214087 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.211671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.7.137.61:33853)
0619 02:45:06.211708 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.211710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.213585 (+ 1875us) server_negotiation.cc:407] Connection header received
0619 02:45:06.213682 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.213683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.213693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.213721 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.214013 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33588: BlockingRecv error: recv got EOF from 127.0.0.1:33588 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:45:06.216266 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.215535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.7.137.61:33853)
0619 02:45:06.215566 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.215568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.215782 (+ 214us) server_negotiation.cc:407] Connection header received
0619 02:45:06.215891 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.215892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.215908 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.215936 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.216226 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33594: BlockingRecv error: recv got EOF from 127.0.0.1:33594 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 02:45:06.219064 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.218351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.7.137.61:33853)
0619 02:45:06.218382 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.218383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.218591 (+ 208us) server_negotiation.cc:407] Connection header received
0619 02:45:06.218686 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.218687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.218697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.218725 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.219026 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33602: BlockingRecv error: recv got EOF from 127.0.0.1:33602 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:45:06.222888 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.222173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.7.137.61:33853)
0619 02:45:06.222204 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.222205 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.222401 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:45:06.222492 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.222493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.222502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.222530 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.222849 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33606: BlockingRecv error: recv got EOF from 127.0.0.1:33606 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 02:45:06.228802 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.228042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.7.137.61:33853)
0619 02:45:06.228074 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.228076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.228272 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:45:06.228368 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.228369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.228378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.228407 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.228708 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33622: BlockingRecv error: recv got EOF from 127.0.0.1:33622 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:45:06.238680 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.237934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.7.137.61:33853)
0619 02:45:06.237977 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.237979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.238184 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:45:06.238278 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.238279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.238289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.238319 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.238626 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33624: BlockingRecv error: recv got EOF from 127.0.0.1:33624 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 02:45:06.256865 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.255968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.7.137.61:33853)
0619 02:45:06.256008 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.256010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.256247 (+ 237us) server_negotiation.cc:407] Connection header received
0619 02:45:06.256350 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.256351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.256362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.256392 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.256811 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33626: BlockingRecv error: recv got EOF from 127.0.0.1:33626 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 02:45:06.290906 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.290208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33640 (local address 127.7.137.61:33853)
0619 02:45:06.290255 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.290258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.290402 (+ 144us) server_negotiation.cc:407] Connection header received
0619 02:45:06.290463 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.290464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.290476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.290533 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.290849 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33640: BlockingRecv error: recv got EOF from 127.0.0.1:33640 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 02:45:06.356640 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.355958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.7.137.61:33853)
0619 02:45:06.356015 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.356017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.356118 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:45:06.356154 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.356155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.356167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.356243 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.356569 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33646: BlockingRecv error: recv got EOF from 127.0.0.1:33646 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 02:45:06.486322 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.485711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.7.137.61:33853)
0619 02:45:06.485785 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.485787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.485824 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:45:06.485904 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.485905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.485915 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.485974 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.486273 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33662: BlockingRecv error: recv got EOF from 127.0.0.1:33662 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:45:06.644301 30615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9553ccf09e46abb6eab5c0a06ec10c" member_type: VOTER last_known_addr { host: "127.7.137.62" port: 36017 } } peers { permanent_uuid: "02f5f7d884b44b098f0a2748d289efe1" member_type: VOTER last_known_addr { host: "127.7.137.61" port: 33853 } } peers { permanent_uuid: "d6c683b91a5a44658460f837e100677e" member_type: VOTER last_known_addr { host: "127.7.137.60" port: 39321 } } }
I20260619 02:45:06.644460 30615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9553ccf09e46abb6eab5c0a06ec10c [sys.catalog]: This master's current role is: LEADER
W20260619 02:45:06.744445 30524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:45:06.743537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.7.137.61:33853)
0619 02:45:06.743588 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:06.743591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:06.743802 (+ 211us) server_negotiation.cc:407] Connection header received
0619 02:45:06.743872 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:06.743874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:06.743889 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:06.743959 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:45:06.744383 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33676: BlockingRecv error: recv got EOF from 127.0.0.1:33676 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 02:45:07.112111 30612 catalog_manager.cc:1259] Loaded cluster ID: 964cd740eda04b5cb7cb631aabdb3936
I20260619 02:45:07.112188 30612 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: loading cluster ID for follower catalog manager: success
I20260619 02:45:07.113011 30612 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: acquiring CA information for follower catalog manager: success
I20260619 02:45:07.113312 30612 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6c683b91a5a44658460f837e100677e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:45:07.182778 30657 catalog_manager.cc:1259] Loaded cluster ID: 964cd740eda04b5cb7cb631aabdb3936
I20260619 02:45:07.182849 30657 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: loading cluster ID for follower catalog manager: success
I20260619 02:45:07.183701 30657 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: acquiring CA information for follower catalog manager: success
I20260619 02:45:07.184015 30657 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02f5f7d884b44b098f0a2748d289efe1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:45:07.260464 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:07.257679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.7.137.61:33853)
0619 02:45:07.257808 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:07.257810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:07.257885 (+ 75us) server_negotiation.cc:407] Connection header received
0619 02:45:07.257958 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:07.257960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:07.258025 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:07.258114 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:07.258264 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.258805 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.259598 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.259843 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.259917 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:07.260092 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:07.260212 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:07.260301 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:07.260340 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 02:45:07.260361 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260619 02:45:07.264212 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:07.262099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.7.137.60:39321)
0619 02:45:07.262208 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:07.262210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:07.262218 (+ 8us) server_negotiation.cc:407] Connection header received
0619 02:45:07.262264 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:07.262264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:07.262297 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:07.262366 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:07.262471 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.262931 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.263589 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.263777 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.263829 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:07.263946 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:07.264045 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:07.264101 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:07.264134 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 02:45:07.264151 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":69,"threads_started":1}
I20260619 02:45:07.268536 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:07.265985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.7.137.61:33853)
0619 02:45:07.266033 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:07.266035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:07.266101 (+ 66us) server_negotiation.cc:407] Connection header received
0619 02:45:07.266173 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:07.266174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:07.266183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:07.266248 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:07.266371 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.267098 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.267898 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.268094 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.268167 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:07.268296 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:07.268333 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:07.268382 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:07.268440 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:45:07.268462 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 02:45:07.269063 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:07.265873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48646 (local address 127.7.137.60:39321)
0619 02:45:07.265909 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:07.265911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:07.266243 (+ 332us) server_negotiation.cc:407] Connection header received
0619 02:45:07.266317 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:07.266318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:07.266330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:07.266383 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:07.266579 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.267249 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.268240 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.268535 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.268591 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:07.268799 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:07.268827 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:07.268855 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:07.268954 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:45:07.268978 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 02:45:07.269501 30742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:45:07.265862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.7.137.62:36017)
0619 02:45:07.266018 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:45:07.266021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:45:07.266073 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:45:07.266152 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:45:07.266153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:45:07.266206 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:45:07.266286 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:45:07.266601 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.267081 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.268205 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:45:07.268513 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:45:07.268569 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:45:07.269124 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:45:07.269244 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:45:07.269308 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:45:07.269397 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:45:07.269423 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260619 02:45:07.271014 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 30377
I20260619 02:45:07.276338 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 30452
I20260619 02:45:07.281422 7716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_8LRmf/build/release/bin/kudu with pid 30527
2026-06-19T02:45:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2768 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5626 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267686 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST